High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Fernando Frediani]

Hi all

First thing I would suggest is ask Netcup to increase the traffic limit. 
Does it cost significantly ? If so how much ? Traffic nowadays is 
normally pretty cheap everywhere in Datacenters.
If that is not an option reducing traffic generated (for example by 
eliminating the SNAPSHOT mirror, or reduce it to fewer places) could be 
an option.
Even finding another provider could be a preferred option I think.

Regarding the suggestion to focus on CDN usage I kind of understand the 
idea, and most of it make sense, however in the other hand mirrors don't 
cost anything (except for this current limitation) and have their value 
in different ways.
Couldn't a sync process for mirrors to fetch data from the CDN instead 
of the server in Germany be something to be considered ? I know it 
wouldn't be done using rsync but perhaps some similar method can be 
developed to achieve that securely.

Another idea in the line of what suggested would be to have one main 
mirror in each country/region and ask other mirrors to sync from them, 
but I would rather explore the above options first.

Regards
Fernando

On 1/6/2026 10:54 AM, Rich Brown wrote:
> Thank you Baptiste, for this analysis.
>
> I re-sorted your list to by data consumption. it’s clear that the mirror traffic is 4x the second place consumer, and 3x ALL the traffic. In fact, the mirror traffic on its own seems to exceed the traffic cap.
>
> - rsync downloads from mirrors : 4 TB / day
> - HTTP downloads.openwrt.org : 1 TB / day (mostly origin traffic for Fastly CDN)
> - HTTP sources.cdn.openwrt.org : 100 GB / day (mostly origin traffic for Fastly CDN)
> - rsync downloads from buildbot workers : 30 GB /day
> - rsync sources : 5 GB /day
>
> I understand (and would support) your multi-tier suggestion below, but I am going to suggest a heretical solution, not because I advocate for it, but simply to start a discussion.
>
> Proposal: We should eliminate all mirrors.
>
> Background: 20 years ago (more or less) mirror download sites for popular projects were essential. Primary hosts didn’t have the horsepower to handle all the download requests and traffic. In addition, it allowed “far flung” institutions to show their camaraderie and support for the project. They were a good thing. Today, in the presence of CDNs and much higher speed hosts, it’s not clear that all this “mirror machinery” is justified by the complexity.
>
> Analysis: We should determine whether these mirrors actually help our central traffic problem. How many mirror sites do we support? Do we know the outbound traffic for In the mirror sites? What kind of data do they provide (sources? binary images?)
>
> It is thinkable to me (although I have no information either way), that our central servers + Fastly CDN could easily handle the requests from individuals who otherwise would go to the mirrors.
>
> If that’s the case, we could eliminate all these mirrors, eliminate the (human) communication and whatever infrastructure we support, save us some bandwidth, and some brainpower.
>
> Of course, I may totally misunderstand the problem, or be proven to be totally wrong - that’s OK. But still I felt I need to ask the question.
>
> Best regards,
>
> Rich
>
>> On Jan 6, 2026, at 06:13, Baptiste Jonglez <baptiste at bitsofnetworks.org> wrote:
>>
>> I think we should move to a 2-tier mirror architecture: have only 2-3
>> tier-1 mirrors that are allowed to pull from us, and then ask all other
>> mirrors to become tier-2 and to pull from a tier-1 mirror.  Tier 1 mirrors
>> should mirror the entire archive and should commit to a long-term
>> relationship.  I can contact mirror operators I know, and if they agree,
>> set things up this way.
>>
>> Of course buildbot workers and the archive server would still be allowed
>> to rsync from the download server.
>>
>
> _______________________________________________
> openwrt-adm mailing list
> openwrt-adm at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-adm