[RFC] Refactoring OpenWrt's build infra

[Petr Štetiar]

Thibaut <hacks at slashdirt.org> [2022-10-05 17:56:17]:

[adding Jo and Paul to Cc: loop]

Hi,

> Before I set on to revamp the system accordingly I want to ask if this
> proposal seems like a Good Idea™ :)

those above mentioned topics are on my TODO list for a long time already, so
any help is more then appreciated, thanks!

Since we're currently using buildbot repository as our main source for the
production containers, I would like to suggest use of issues[1] there to track
future plans and ongoing work transparently over there for obvious reasons.
Other option might be mirroring of that GitLab buildbot repo to GitHub and use
issues there instead if thats preferred.

More food for thoughts:

 * We should replace currently HW EOL machine serving buildbot.openwrt.org

   - we're currently blocked with this by still pending OpenWrt.org account on Hetzner
   - this refactoring might be a good opportunity for tackling it

 * Filter out GitPoller build events originating from noop sources like the CI tooling[2]

   - IIRC those build events gets propagated down to 2nd stage/package builds as well

 * Rate/resource limits handling during scripts/feeds invocations

   - git.openwrt.org might be overloaded in certain time periods, leading to
     waste of build resources and false positive build results

     * python3/host: build install race condition with
       uboot/scripts/dtc/pylibfdt[3] is another such resource waste example

 * Use HSM backed storage for release/package signing keys

 * IIRC Paul (and probably more folks) find our buildbot based system arcane
   and would like to try using something more recent, like for example GitHub
   Actions instead

   - perhaps we should try to align with those ideas and consider factoring
     out the build steps into something more self-contained, build layer
     agnostic and thus reusable?

   - it just seems to me, that we're reinventing a wheels[4]

 * We should consider making our buildbot infra completely open, so anyone can
   reuse it and/or make it better

1. https://gitlab.com/openwrt/buildbot/-/issues/new
2. https://github.com/openwrt/openwrt/pull/10094#issuecomment-1170760326
3. https://github.com/openwrt/openwrt/pull/10407
4. https://github.com/openwrt/packages/issues/19241

Cheers,

Petr