Important changes to rsync mirroring from OpenWrt download server

[Baptiste Jonglez]

On 09-01-26, Fernando Frediani wrote:
> Was it asked to Netcup the possibility  to increase the bandwidth available
> than it is now ?

Yes, and it does not seem possible.  Besides, they only reset the
rate-limit once per month.  We were lucky that the monthly period starts
on the 9th, so we were unblocked today.

Apparently their rule is: after 120 TB of monthly data, and if exceeding
1 Gbit/s for 60 minutes, you become rate-limited to 200 Mbit/s for the
rest of the monthly period.

We want to avoid this rate-limit in the near future, hence the limitation
on rsync.  Finding a new home for the main download server is also on the
table as a longer-term solution.

In any case, controlling the number of mirrors is always beneficial (even
with unlimited bandwidth, we would end up being limited by disk I/O, CPU...)

Baptiste

> On 1/9/2026 1:49 PM, Baptiste Jonglez wrote:
> > Hi again,
> > 
> > We unfortunately had to close the rsync server starting from today to get
> > back to manageable traffic levels.  Only a few mirrors identified as
> > potential tier-1 mirrors are allowed to sync, the list is available here:
> > 
> >    https://openwrt.org/downloads#mirrors
> > 
> > Beware that this list may change in the near future (new entries / removed
> > entries), and not all listed mirrors provide rsync service.
> > 
> > We already got several proposals for tier-1 mirrors, thank you, we will
> > get back to you.
> > 
> > Baptiste
> > 
> > On 07-01-26, Baptiste Jonglez wrote:
> > > Hello,
> > > 
> > > Up to now, we have been offering an open rsync service to allow anybody to
> > > mirror OpenWrt images and packages.  Due to the increasing number of
> > > people using this rsync service, this is becoming unsustainable and starts
> > > to affect more critical uses of the main OpenWrt download server.
> > > 
> > > As a first step, we have just disabled mirroring of snapshot builds
> > > through rsync.  The existing target "rsync://rsync.openwrt.org/downloads"
> > > now only offers release builds.
> > > 
> > > We believe snapshot builds are not very useful to mirror, and mirroring
> > > them generates a lot of traffic since they are rebuilt continuously.
> > > However, if you really have a good use-case for mirroring snapshot builds,
> > > you can find them under the new target "rsync://rsync.openwrt.org/downloads-snapshots".
> > > Please pull every 24 hours at most, or even only once a week if it fits
> > > your use-case.  You can also selectively mirror only the targets you need.
> > > If we notice too much traffic, we might close this new rsync target.
> > > 
> > > 
> > > As a second step, we will soon introduce a mirror tiering system, and only
> > > allow a few designated "tier 1" mirrors to pull from the main download
> > > server.  All other mirrors will be considered "tier 2" and will be
> > > required to sync through a tier-1 server.
> > > 
> > > Anybody can apply to become a tier-1 mirror, we are looking at good
> > > geographical distribution throughout the world.  We are still working out
> > > the details, here is a tentative list of requirements (open to discussion):
> > > 
> > > - good uptime track record
> > > - good network connectivity (ideally 1 Gbit/s of dedicated capacity)
> > > - IPv6 connectivity
> > > - unlimited outgoing traffic volume
> > > - commitment to host OpenWrt data for at least several years
> > > - carry all releases (4 TB currently, provision 6 TB for future growth)
> > > - serve OpenWrt content over HTTP, HTTPS, and public rsync
> > > - monitoring to detect failed rsync runs or expired HTTPS certificate
> > > - a responsive contact email
> > > 
> > > We will provide recommended rsync setup.
> > > 
> > > Optionally, we are also exploring the possibility to point our
> > > downloads.openwrt.org CDN to use these new tier-1 mirrors as origin.
> > > It would probably only be done in case of emergency. If you are open to
> > > such a thing, please let us know.
> > > 
> > > To apply, you can post publicly on the openwrt-adm at lists.openwrt.org
> > > mailing list or reply privately to me (my first name at bitsofnetworks.org).
> > > 
> > > Baptiste
> > 
> > 
> > > _______________________________________________
> > > openwrt-devel mailing list
> > > openwrt-devel at lists.openwrt.org
> > > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
> > 
> > _______________________________________________
> > openwrt-devel mailing list
> > openwrt-devel at lists.openwrt.org
> > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
> 
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel