High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Baptiste Jonglez]

Hi,

The current download server is donated by Netcup in Germany [1].  It has
been working well for some time, but we are now hitting traffic limits.
According to their conditions, after 3 TB / day, we are rate-limited to
300 Mbit/s.  This has been causing issues with some buildbot workers that
time-out when downloading the SDK to build packages.

I see three solutions:

- ask Netcup to increase the traffic limit
- reduce the amount of traffic we generate
- find a new hosting provider for the download server

One of the problem is that we mix many usages on the same server, some
critical (main HTTP downloads, buildbot workers pulling the SDK) and some
less critical (mirroring to third-parties).

I did some stats, here is the average amount of traffic we sent *per day*
from the download server in December 2025:

- HTTP downloads.openwrt.org : 1 TB / day (mostly origin traffic for Fastly CDN)
- HTTP sources.cdn.openwrt.org : 100 GB / day (mostly origin traffic for Fastly CDN)
- rsync downloads from mirrors : 4 TB / day
- rsync downloads from buildbot workers : 30 GB /day
- rsync sources : 5 GB /day

As you can see, we have many mirrors pulling from the main download
server, with some mirrors in particular generating a lot of traffic (up to
300 GB / day for some mirrors).

I think we should move to a 2-tier mirror architecture: have only 2-3
tier-1 mirrors that are allowed to pull from us, and then ask all other
mirrors to become tier-2 and to pull from a tier-1 mirror.  Tier 1 mirrors
should mirror the entire archive and should commit to a long-term
relationship.  I can contact mirror operators I know, and if they agree,
set things up this way.

Of course buildbot workers and the archive server would still be allowed
to rsync from the download server.

Any thoughts about this?

Baptiste

[1] https://openwrt.org/infrastructure
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openwrt.org/pipermail/openwrt-devel/attachments/20260106/12a6f01c/attachment.sig>