New CDN via fastly.com
baptiste at bitsofnetworks.org
Sat Jan 23 05:36:26 EST 2021
On 22-01-21, Paul Spooren wrote:
> > Like jow, I don't see the point of putting an OpenWrt-managed CDN in
> > front
> > of all project-specific mirrors.
> Where did he say that, I only have a incomplete record of the adm mails
That's in the commit message you linked above: https://git.openwrt.org/bf96eb55c8
> > Think about it by assuming you pay for the CDN usage. Why would you pay
> > to serve data to all buildroot users when there are already many
> > different
> > mirrors to handle this traffic? Also, it centralizes source downloads
> > while there is no need to do it.
> > It's fine to add it as a fallback like we do currently with
> > sources.openwrt.org. In both cases, the sources.openwrt.org origin
> > server
> > is still a single point of failure.
> I described my problem a while ago when we used KeyCDN:
> I'm currently downloading `curl` from a mirror with 3kB/s. This is
> likely not the fault of uxnr.de but of a bad fiber connection from
> Hawaii to Germany. This throughput reminded me of this email and I'd
> like to ask again if there are any reasons against offering
> sourcescdn.openwrt.org as a CDN for sources.openwrt.org.
> For testing I set it up at https://sourcescdn-1212d.kxcdn.com and it
> works as expected. With a modified download.pl I now receive sources
> with 200Mbit/s.
> This doesn't work as a fallback as extremely low downloads speeds are not
> considered as "failure".
Then you can just change the mirror order in your own tree. Or the
download function needs to be more intelligent. Or talk with the mirror
As I said, I don't think it's acceptable that the OpenWrt project handles
the main download load for non-openwrt sources. Sponsorship is always
delicate: at some point, CDN sponsorship might stop, and we shouldn't pay
for other people's traffic. The previous CDN experience highlights just
that: if I understand correctly, they terminated the sponsorship because
the source CDN was generating too much traffic for them.
> > > Due to the recent buildbot issues with updating feeds I'm wondering
> > > if we
> > > should add all standard feeds as folders to sources.o.o and change
> > > the
> > > default feeds file to download from the CDN rather than git.o.o.
> > > This should
> > > give a massive release to the git servers and future failures.
> > >
> > > Additionally we could use a non-public origin server which feeds
> > > the CDN and
> > > mirrors and distribute downloads.o.o directly via CDN.
> > For downloads.openwrt.org, the CDN service should be equivalent to the
> > current service. Downloads from device should not be automatically
> > redirected to HTTPS (for opkg packages, and also for sysupgrade
> > downloads
> > from the device itself). I think this is currently handled through
> > user-agent matching.
> The sources.c.o.o is set to serve whatever is asked for, no magic headers
> needed. I'd implement the same mechanism for downloads.c.o.o, no redirects.
I don't see the point about the sources mirror, I'm talking about the
download server. Adding a CDN should provide equivalent service to what
we have today, otherwise this is a regression.
> > And it should definitely have IPv6.
> Should be solved.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: not available
More information about the openwrt-adm