High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Hauke Mehrtens]

On 1/6/26 12:13, Baptiste Jonglez wrote:
> Hi,
> 
> The current download server is donated by Netcup in Germany [1].  It has
> been working well for some time, but we are now hitting traffic limits.
> According to their conditions, after 3 TB / day, we are rate-limited to
> 300 Mbit/s.  This has been causing issues with some buildbot workers that
> time-out when downloading the SDK to build packages.
> 
> I see three solutions:
> 
> - ask Netcup to increase the traffic limit
> - reduce the amount of traffic we generate
> - find a new hosting provider for the download server
> 
> One of the problem is that we mix many usages on the same server, some
> critical (main HTTP downloads, buildbot workers pulling the SDK) and some
> less critical (mirroring to third-parties).
> 
> I did some stats, here is the average amount of traffic we sent *per day*
> from the download server in December 2025:
> 
> - HTTP downloads.openwrt.org : 1 TB / day (mostly origin traffic for Fastly CDN)
> - HTTP sources.cdn.openwrt.org : 100 GB / day (mostly origin traffic for Fastly CDN)
> - rsync downloads from mirrors : 4 TB / day
> - rsync downloads from buildbot workers : 30 GB /day
> - rsync sources : 5 GB /day
> 
> As you can see, we have many mirrors pulling from the main download
> server, with some mirrors in particular generating a lot of traffic (up to
> 300 GB / day for some mirrors).
> 
> I think we should move to a 2-tier mirror architecture: have only 2-3
> tier-1 mirrors that are allowed to pull from us, and then ask all other
> mirrors to become tier-2 and to pull from a tier-1 mirror.  Tier 1 mirrors
> should mirror the entire archive and should commit to a long-term
> relationship.  I can contact mirror operators I know, and if they agree,
> set things up this way.
> 
> Of course buildbot workers and the archive server would still be allowed
> to rsync from the download server.
> 
> Any thoughts about this?
> 
> Baptiste
> 
> [1] https://openwrt.org/infrastructure

Do we need mirrors for the snapshots at all?

Maybe the mirrors could stop mirroing the target snapshots to reduce the 
traffic.

If we loose the snapshot we would just rebuild them.

Hauke