High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Baptiste Jonglez]

On 06-01-26, Thibaut wrote:
> 
> 
> > Le 6 janv. 2026 à 15:27, Baptiste Jonglez <baptiste at bitsofnetworks.org> a écrit :
> > 
> > I like provocative thinking ;)
> 
> Then here’s another thought :)
> 
> > One thing that is maybe not clear: we don't manage the mirrors ourselves,
> > and the mirrors are not used by downloads.openwrt.org.  All "official"
> > traffic is already handled by the CDN and our single download server, with
> > old releases handled by the archive server.  So, mirrors don't cost us
> > much in term of time and infrastructure (except this traffic problem).
> > So far, the machinery is just "the rsync server is open to all".
> > 
> > There are two kind of mirrors as far as I know: "far-flung institutions
> > showing their support" as you say, and people running private mirrors
> > (which I guess they have good reasons for, maybe they have large fleets of
> > OpenWrt devices)
> 
> About that. I can give you an excellent reason to run a local mirror when you depend on packages repositories being available and « working », e.g. when frequently building lots of images using IB: when the CDN becomes out of sync (say Packages doesn’t match the actual files available for download - and it happens quite often), you’re screwed.

This is a separate issue we should fix.  I believe we can just disable CDN
caching for packages and Packages* files, and keep caching only for images
and SDK/IB.  I will try this.

> As for the crazy amount of traffic, as mentioned on IRC I believe it’s partly due to our (honestly broken) way of updating the packages repositories: updating the entire archive whenever a single commit that affects a single package is pushed cannot scale.
> 
> I believe your proposal simply delays the inevitable: it will push down this problem to tier-1 mirrors but eventually we will be caught back again as the number of packages and platforms grow.
> 
> I believe it would make more sense to work on having a saner build system that can (build - that would also save CPU cycles - and) update only actually modified packages. There’s been preliminary work in phase1 to address this (at least for the ‘update’ part). Then all forms of mirror traffic should naturally decrease as untouched bits remain valid in cache.

Agreed but this is a much bigger undertaking.

As suggested by Paul and Hauke, we will exclude snapshots from rsync
mirroring as a first step, this should already help quite a lot (well,
"delay the inevitable" as you say ;) ).

The only thing that will be constantly updated by mirrors are packages for
the active stable releases, typically the last two major releases.

Baptiste