[PATCH opkg 0/5] Purge packages from cache when they have incorrect checksum

[Paul Spooren]

On Fri Nov 20, 2020 at 12:43 AM HST, Baptiste Jonglez wrote:
> Hi,
>
> Any news on this patch series? It should definitely be considered
> before the 20.XX branching because it fixes FS#2690 (bug related to
> imagebuilder), and I would like to backport it to 19.07 at some point.

I CI built this and tested the ImageBuilder as follows:

* Run once and create firmware images
* Run `echo "foo" >> dl/base-files*.ipk
* Run again

The `opkg` detects correctly a checksum missmatch and downloads a fresh
copy again. Excellent! Below a snipped log

--- 8< ---
Installing kernel (5.4.77-1-211d098873122e25225ddfe3f75dbf13) to root...
Configuring kernel.
Installing base-files (237-r14984-d369993898) to root...
Removing base-files from cache because it has incorrect checksum.
Downloading http://downloads.openwrt.org/snapshots/targets/ath79/generic/packages/base-files_237-r14984-d369993898_mips_24kc.ipk
Installing libubox20191228 (2020-08-06-9e52171d-1) to root...
Copying /tmp/f1/openwrt-imagebuilder-ath79-generic.Linux-x86_64/dl/libubox20191228_2020-08-06-9e52171d-1_mips_24kc.ipk.
Installing libuci20130104 (2020-10-06-52bbc99f-3) to root...
--- >8 ---

I couldn't come up with a simple way to change the size while keeping the same
checksum (hoping overall there is no such way anyway), so that's untested.
However, as this patch "recycles" the previously working size check I have no
doubt it keeps working.

Size for mvebu/cortexa9/arm_cortex-a9_vfpv3-d16 increased a tiny bit, but that
seems totally worth it.

	Comparing package sizes...
	Change 	Local	Remote 	Package
	+98	66602	66504	opkg

Therefore,

Acked-by: Paul Spooren <mail at aparcar.org>

>
> Thanks,
> Baptiste
>
> On 25-08-20, Baptiste Jonglez wrote:
> > From: Baptiste Jonglez <git at bitsofnetworks.org>
> > 
> > The motivation of this patch series is to fix FS#2690.  Because packages
> > are continuously rebuilt, the ImageBuilder ends up with old packages in
> > its cache, and fails because checksums don't match with the new package
> > index from the download server.
> > 
> > The approach to solve this problem is the following.  Before using a package
> > from the cache, verify its size and checksum against the package index, and
> > delete the package from the cache if they don't match.  The install process
> > will then proceed to download the "fixed" package as usual.
> > 
> > The main patch is "download: purge cached packages that have incorrect
> > checksum".  The other ones are either cleanup or refactoring to prepare
> > for the main change.
> > 
> > Baptiste Jonglez (5):
> >   download: remove compatibility with old cache naming scheme
> >   libopkg: factor out checksum and size verification
> >   download: factor out the logic for building cache filenames
> >   download: purge cached packages that have incorrect checksum
> >   opkg_verify_integrity: better logging and error conditions
> > 
> >  libopkg/opkg_download.c | 123 ++++++++++++++++++++++++++++++++--------
> >  libopkg/opkg_download.h |   1 +
> >  libopkg/opkg_install.c  |  76 +++----------------------
> >  3 files changed, 108 insertions(+), 92 deletions(-)
> > 
> > -- 
> > 2.27.0
> > 
> > 
> > _______________________________________________
> > openwrt-devel mailing list
> > openwrt-devel at lists.openwrt.org
> > https://lists.openwrt.org/mailman/listinfo/openwrt-devel