OpenWrt Next Generation Ideas

[Arınç ÜNAL]

On 15.04.2023 20:02, Sam Kuper wrote:
> On Fri, Mar 31, 2023 at 12:44:12PM +0300, Arınç ÜNAL wrote:
>> These are the ideas I've been thinking about for the future of OpenWrt
>> for a while. It looks complete enough to share it with all of you.
>> [...]
>>
>> https://arinc9.notion.site/OpenWrt-next-gen-ideas-6db745e7584b4823950291c96f2326bb
> 
> Please can you post the contents of that page as an email in this
> thread?  Doing so will yield the following advantages:
> 
> 1.  People (like me) whose browsers/OSes are set up to accommodate
>      accessibility issues will be able to read it.
> 
>      (Currently, if I visit that URL I only get a blank page.  Maybe that
>      is fixable in some way but it would probably take a lot of work,
>      whereas a plain text email would be completely accessible to me.)
> 
> 2.  People reading the mailing list archives will be able to make sense
>      of the discussion, even if the URL above has succumbed to entropy in
>      the meantime.

Sure. Here's a markdown version below. Keep in mind you did not send
this mail to me, I've only seen it by manually checking the mailing list.

# OpenWrt next-gen ideas

## Contribution to the Linux Kernel

- Contribute defconfigs and all the devicetrees on OpenWrt to Linux.
     - Create a Linux branch for that and maintain it.
     - Submit pull requests to the Linux mailing list regularly to get them applied.
- Either submit all out-of-tree patches on OpenWrt to Linux or get rid of them and find a better solution for what the unacceptable patch does.
- Architecture specific patches for Linux should be about backporting DTs and defconfigs.
     - Bugfix backporting should happen only after it's accepted to Linux. The patch must be identical to the commit on Linux.
     - Feature backporting should be done only if it's thoroughly tested.
     - An issue caused by feature backporting.
         
         [https://github.com/openwrt/openwrt/issues/9420](https://github.com/openwrt/openwrt/issues/9420)
         

## OpenWrt Kernel Configuration Solution

- defconfig for each device instead of config for each (sub)target.
     - Should lower kernel size greatly.
     - OpenWrt provides own kernel module packages. The SDK must disable the kernel symbol on the defconfig which matches the enabled kernel module on OpenWrt SDK before compiling.
         - Instead of this, just `make target_defconfig && make mod2noconfig`, then compile normally with kernel module packages. This way, OpenWrt compiles a kernel with the least amount of kernel modules (or rather, it compiles the kernel like it always did), and people compiling the kernel outside of OpenWrt can choose to remove the kernel modules they don't need.
     - defconfig for each (sub)target is much easier to maintain.
         - The most basic options and network drivers should be built into the kernel. The rest should be selected as kernel modules.

### Incorporating defconfigs with OpenWrt's kernel config system

- Treat defconfigs as the generic kernel config. Only keep (sub)target configs on OpenWrt. These configs would do changes specific to OpenWrt.
     - Booting the image, `CONFIG_CMDLINE`, etc.
     - Filesystem solution, `CONFIG_UBIFS_FS`, `CONFIG_SQUASHFS`, `CONFIG_JFFS2_FS`, etc.
     - Non-mainline drivers.
- First, turn the defconfig to a full config with `make target_defconfig`, like the generic config is. Then, apply the target config.
- This should greatly simplify the kernel configs on OpenWrt and grant people outside of the OpenWrt environment to compile the kernel for their device.
- For submitting defconfigs to mainline Linux, we take out the OpenWrt-specific things from the config.
- Similar to the current process of moving kernel symbols from the target config to the generic kernel, move the kernel symbols to .config, run `make savedefconfig` and move the defconfig file back to `arch/*/configs/target_defconfig`, then submit a patch which updates the defconfig file to our OpenWrt Linux submissions repository.
- Implementation should happen progressively. Put the defconfig on Linux, backport it on OpenWrt, start using the defconfig instead of the generic config, specifically on that (sub)target.
- Quotes
     - The explanation of OpenWrt's kernel config system from Felix Fietkau.
     
     > I believe the current OpenWrt kernel config system ist vastly superior for keeping target kernel configs in sync while at the same time making it easy to keep an overview of target specific overrides as well. This is especially relevant for updating the kernel version.
     >
     >
     > We do have a lot of config options that we set as default for all
     > targets. With our current system, we keep them in a file in
     >
     > target/linux/generic/config-<version>
     >
     > A target can override any of those with its target specific config
     > overlay. The clever bit of our kernel config system is that this overlay
     > is not handwritten. You can use make kernel_menuconfig or
     > kernel_oldconfig to make changes to it, and it will automatically filter
     > out the defaults from the generic config.
     >
     > When updating to a newer kernel for the first time, you can pick any
     > target, copy the old version and run make kernel_oldconfig. Afterwards
     > you can look through the diff and move any newly added items that should
     > be generic over to the generic config template, so you won't be asked
     > about it for the next target.
     >
     > The expectation is that target configs should share as many options as
     > possible, so it helps that you can simply go through the target config
     > overlays and look at each line to see if it should stay target specific
     > or if it should be moved to the generic config instead.
     >
     > Also, OpenWrt commits that change settings for all targets are much
     > easier to review compared to a set of full configs, because you don't
     > have to manually verify if you properly applied them to all targets.
     >
     > It would definitely be possible to build a workflow around full configs
     > as well to try to achieve something similar, but the problem that I see
     > there is that the relevant information for maintaining would not be as
     > readily available. And it would be easier for configs to accidentally
     > drift apart, causing spurious build issues when dealing with the
     > configurability of our module packages.
     >
     
     [Some Modest Virtualization Observations](http://lists.openwrt.org/pipermail/openwrt-devel/2023-March/040712.html)
     
     > Yet this loses some of the most important benefits of OpenWRT and VMs.
     The drivers for*all* hypervisors are built-in, instead of merely loading
     the appropriate ones as needed.  Hence the recommendation of 256MB of
     memory, instead of the 128MB recommended for device purchases.  Most
     hypervisor machines can handle this, but let the small VMs run loose and
     you'll be out of memory instantly.
     
     This fails as a developer simulcrum for an embedded device since too much
     memory is consumed to support all devices.  Using modules for portions of
     the hypervisor support should cut down significantly.
     
     The kernel configurations also need work.  Notice CONFIG_FB=y?  For a
     router/AP you really don't need that feature.  I suppose someone might
     need that in some circumstance, yet for production use that is major
     overhead.
     
     Speaking of support for all hypervisors.  I notice
     "# CONFIG_XEN_BACKEND is not set".  Most Xen device backends might be
     unnecessary, but CONFIG_XEN_NETDEV_BACKEND certainly seems worthwhile.
     >

## Kernel and Filesystem Compilation Solution

- Make a mode menu.
     - Filesystem only.
         - OpenWrt only sets up a filesystem image.
         - Use the kernel headers from the current kernel.
     - Full image.
         - OpenWrt compiles longterm kernel, sets up filesystem image, combines them together, makes the image bootable by the bootloader of the device, U-Boot, CFE, etc.
         - OpenWrt provides own kernel module packages. Make kernel module selection available to use only when the full image mode is selected.
             - When installing kernel modules using opkg, the kernel hash won't match on other kernels so this part is covered.
- Make a kernel selection menu where the user can choose to feed the kernel directory of their own or use the longterm one defined on the OpenWrt SDK. Add this as a suboption to the full image mode.
     - Developer option. Some parts of the process, LZMA compression, etc, may not work depending on the kernel size.
     - `CONFIG_EXTERNAL_KERNEL_TREE` is similar.

Arınç