High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Michael Richardson]

Baptiste Jonglez <baptiste at bitsofnetworks.org> wrote:
    rich> Proposal: We should eliminate all mirrors.

If the mirrors are ad-hoc, and are not listed as alternate A/AAAA RR, then
we might want to artificially rate-limit the rsync support.  Are there some
pathologies here?  Peers that repeated pull the same content?

    > One thing that is maybe not clear: we don't manage the mirrors ourselves,
    > and the mirrors are not used by downloads.openwrt.org.  All "official"
    > traffic is already handled by the CDN and our single download server, with
    > old releases handled by the archive server.  So, mirrors don't cost us
    > much in term of time and infrastructure (except this traffic problem).
    > So far, the machinery is just "the rsync server is open to all".

In the past, I've mirrored many things locally because I found that my builds
failed with timeouts.

    > There is an incomplete list here: https://openwrt.org/downloads#mirrors

    > I think public mirrors can still be useful, it leaves alternative choices
    > if we have a big outage of the download server and the archive server
    > (unlikely, but still possible).  Another reason is better performance: the
    > CDN helps but still needs to fetch content from the download server in
    > Germany.

...

    > That being said, I agree the current number of mirrors is kind of
    > unreasonable for this purpose.

split out the uses onto different names/IPs (v6 makes this easy).
I assume all the CDNs can use v6.

    > The idea of the 2-tiers system is precisely to delegate mirror
    > distribution to a few trusted people/org that have the knowledge and
    > infrastructure for this.

Works for me.

--
Michael Richardson <mcr+IETF at sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://lists.openwrt.org/pipermail/openwrt-devel/attachments/20260106/1e3e81d4/attachment.sig>