High volume of traffic on download server - move to a 2-tiers rsync setup for mirrors?

[Eric]

On Wednesday, January 7th, 2026 at 06:04, Baptiste Jonglez <baptiste at bitsofnetworks.org> wrote:
> On 06-01-26, Eric wrote:
> > got detection mechanisms on the server that look for out-of-sync package
> > issues and then tell users "wait a while and try again"... I don't know
> > how many dozens (hundreds?) of times I've had to explain this on the forums
> > over the past couple of years as ASU gains more traction with users.
> 
> 
> I believe this is caused by the CDN cache that introduces inconsistencies
> between Packages list and actual packages.


Hi Baptiste,

That makes sense.  I had been thinking it was rsync just being slow, as in:
'busybox*ipk' updates first, then 'Packages' last, and the ASU server is
doing 'opkg install busybox ...' right in the middle of that.  CDN adding
even more latency, well...

> I checked and we are actively busting the CDN cache through the Fastly API
> when updating data on the download server. This is done since at least
> August 2025 (but I don't know if it works well).
> 
> Have you experienced inconsistencies since August?
> 
> If this is a different issue, do you have a link with a description of the problem in details?

I'm unclear on the details, as I have no real insight into what's going on
on the servers, and I have to guess what's going on from user reports.
My suspicion is that people get acclimated to the errors and don't report
them, so I only see the tip of the iceberg.  In any case, I've got a PR 
https://github.com/openwrt/asu/pull/1522 that would give me a better
view of how widespread this is, but I need to on get Paul to merge it.

And yes, we are still seeing reports since Aug, here are a couple from
a quick search of the forum:
https://forum.openwrt.org/t/notice-problems-with-custom-firmware-image-creation/224829/76
https://forum.openwrt.org/t/luci-attended-sysupgrade-support-thread/230552/337

Eric