[RFC] Stop providing binary package updates for release builds?

[Jo-Philipp Wich]

Hi,

since the release of LEDE 17.01.0, OpenWrt started offering updated binary
packages for released versions, means the HEAD of a released stable branch is
continuously getting rebuilt and the resulting binaries are uploaded to the
release repositories. Users will see those updated packages as "upgrades" and
are able to install them via opkg or LuCI.

There were several motivations to work towards providing these "continuous"
updates:

 - Ability to quickly push security or bug fixes without requiring users to
   wait for the next release and forcing them to fully reflash their
   deployments (also having to accept unwanted Kernel updates, potential
   regressions or potential flash failures)

 - Ability to de-duplicate required disk space, no need to keep a ~90%
   identical copy of the entire package universe for each minor release

 - Ability for package maintainers to push important/useful updates to
   stable releases without needing to wait for the next release cycle to
   have these changes available for a wider, non-snapshot using audience

However after over four years of providing these updates it appears that the
overall usefulness of this approach to the user base is not as certain:

 - The wiki actively discourages users from upgrading packages [1]

 - The overall sentiment in the forum is to instruct users to not/never
   upgrade packages [2] but to always upgrade to the next release, build from
   source [3] or use snapshots/IB

 - Many contributors are not aware of the additional responsibilities and
   potential pitfalls when backporting features, fixes or updates to stable
   branches [4]

 - Many developers or active community members do not appear to care about
   binary releases at all, only using the full buildroot to build images
   completely from source

 - In my view, an overall sentiment among most contributors that OpenWrt is a
   source based distribution, with precompiled images and binary package
   repositories being an after thought at best

Given these facts I would like to propose rolling back continuous package
build changes and revert mostly to the pre-LEDE 17.01.0 approach for future
release processes. In particular this entails:

 - Stop providing binary package updates; build release images and associated
   repositories once only, archive the artifacts and redelegate build
   resources back to master snapshots

   - Maintainers must ensure that packages/device images that should be part
     of a release build properly during the RC phase, whatever is broken will
     not be part of the repositories but might/might not reappear with the
     next maintenance release

 - Stop dividing the build process into distinct image and package phases,
   simply build the entire release once from a complete source tree

 - Stop sharing package repositories among different targets

 - Discontinue release branch HEAD builders and only do branch build testing
   during the pre- and RC build phases

Once implemented, this would allow us to:

 - Massively reduce the complexity of the build logic, it will basically
   become a "make world" with no need to track changes, iterative cleanups
   etc.

 - Reduce the amount of needed compute resources, we only need to build master
   snapshots and occasionally ramp up resources to build release images

 - Focus on source code development and master branch features without having
   to worry about binary package peculiarities such as ABI- and feature
   compatibility, versioning or dependency handling

 - Simplify release branch maintenance, build images every few months and in
   case of security issues in vital components like openssl or dropbear we
   could instruct users to wait for the next maintenance release or switch to
   snapshots

 - Optimize packages. Ship different package feature selections for each
   target, e.g. small ones for restricted targets, big fully featured packages
   with hand written assembly optimizations for mvebu. Full coreutils, glibc
   for x86 builds

Potential alternatives for users:

 - Attended sysupgrade instead of releases and package updates to reflash
   master snapshots with updated packages from time to time

 - Build from source

 - Use another downstream distribution


In my opinion, there is a fundamental decision to be made on whether we would
like to pursue the goal of (also) being a binary distribution for
routers/embedded appliances or if we simply concentrate on the build-from-
source aspect and leave the binary package business to forks or other
downstream distributions projects.

Solutions like Debian/Alpine chroots or entware maybe could also be an
alternative for OpenWrt's own binary package ecosystem.

Given that maintaining binary package distributions is hard and boring work we
need to decide whether we want to fully and properly pursue this goal or
rather not at all and focus on always porting master to the latest minor
Kernel version and finding the optimal compilation flags for each SoC
revision.

I am interested in hearing you thoughts and comments on this matter.

Kind regards,
Jo


1: https://openwrt.org/meta/infobox/upgrade_packages_warning
2: https://forum.openwrt.org/search?q=%22never%20upgrade%22%20order%3Alatest
3: https://forum.openwrt.org/search?q=%22build%20your%20own%22%20order%3Alatest
4: https://bugs.openwrt.org/index.php?do=details&task_id=4159


PS: Some points may be intentionally exaggerated.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openwrt.org/pipermail/openwrt-devel/attachments/20211212/e55a5d85/attachment.sig>