Objective of OpenWRT/x86?

[Philip Prindeville]

> On Apr 26, 2023, at 2:17 PM, Elliott Mitchell <ehem+openwrt at m5p.com> wrote:
> 
> Well, was a specific objective ever chosen for the x86 version of
> OpenWRT?


Does it need one?  As the most ubiquitous hardware out there for many users, why would we need to box it in?

Someone might want to throw a generic image onto a PC they have lying around until they decide it's worth bringing up specialized hardware... going out an buying it, building an image, installing it, troubleshooting it, etc.


> I can state my goal/hope for OpenWRT/x86.  The *WRT Linux distributions
> were so named for originally targeting the LinkSys WRT54G.  This was a
> small AP, so one might expect builds to be for small APs.  By today's
> standards 128MB RAM and 16MB of storage is relatively small.


Yeah, we had a discussion some years ago about "FAT" vs. "SKINNY" platforms that never went anywhere.  All of the AMD64 platforms were FAT.

Having robust hardware that you can prototype on, including CPU or memory hungry development, allows us to prove the value of new capabilities that might be ubiquitous in the next generation of COTS and SoHo routers.

After all, the BoM for an extra GB of DRAM is literally pennies in large production volumes.

BTW, I run OpenWrt for its firewall capabilities... None of my boxes actually has wireless hardware at this point, and I use a few Ubiquiti AP-AC-Pro's or U6-LR's scattered around the house.

The rule of thumb when I worked at MIT on the X Window System, was that what's the top-of-the-line workstation today will be "budget" hardware 3-4 years out, so to not worry about hardware constraints because by the time you release stable, production quality software, the hardware will have gone through one or two evolutions...

Today's one-off advanced prototyping platform is tomorrow's BestBuy budget/clearance item...



> Since this network is in the moving towards the server phase, the AP drew
> my attention.  Turns out all of the hardware of an AP is available for
> servers, though in distinct form-factors.  If the server has a full
> IOMMU, the hardware can be directed to a VM and you have a proper AP in a
> VM.


"Moving towards the server phase"?  I'm not getting what you mean by this.

See above: the radios and antennae I can get as add-ons for a Xeon-D 1U pizza box or even an APU6 mPCIe slot are vastly inferior to what ODM purpose-built hardware like an U6-LR can do in terms of cost and performance.

Um... you can't "virtualize" WiFi in any VM I've ever seen.


> Problem is instead of the recommended 128MB memory, 16MB of storage
> (https://openwrt.org/supported_devices/432_warning) the virtualization
> examples (https://openwrt.org/docs/guide-user/virtualization/start) are
> suggesting massively more memory.  256MB (small Xen example), 512MB
> (VMware OpenWRT/15.05) or 1GB (Qemu examples).


Sorry, why is this a "problem"?

I spent $1100 on a Xeon-D box with 128GB of DRAM, 16 hyper threaded cores, and 2TB of NVMe.


> I don't yet have a full handle on the issues.  My first thought was the
> large sizes come from early stage development and not wanting to bother
> with memory limitations.  Another possibility is this comes from simply a
> thought pattern of "x86 memory is cheap, just throw memory at it".


You're looking at it backwards.

Think of it as "what could I do if I have more RAM and CPU?"

Could I do on-board real-time data reduction of firewall logs?  Could I run fail2ban near-realtime?  Could I run a sandboxed Apache proxy to all of my internal HTTP-based services and used certificate-based authentication with centralized management?

I've been running VoIP on my firewall for 12 years now, with find-me follow me, and 22 extensions.  I'm thinking of adding 4K video-conferencing capability if I can find the time...

My security cameras do H.265 in their highest quality mode, but the renderer in Safari on my iPhone can only handle H.264... but I don't care, because I can configure a transcoding proxy to run in Apache+VLC!!!


> Yet if one is wanting to use this for serious purposes, memory IS a
> concern.  GCC is pretty capable for x86, a quick check suggests GCC tends
> to produce binaries for x86 which are four-fifths the size of those for
> ARM.  Yet everything for OpenWRT/x86 is suggesting at least *double* the
> memory?!


If I need something for "serious purposes", I budget appropriately...


> One issue I've found is the kernel configurations seem ill-suited to x86.
> Almost any storage driver used by 10 or more people is built into the
> kernel.  As a result the *single* kernel is huge.


If it's not used as a boot device, we could make it kmod-able... otherwise we'd need to add initramfs...  I don't think anyone wants to go down that road.  Too easy to brick devices.

I think we should leverage more subtargets and profiles, but that's a separate discussion.



> The conventional approach for x86 is to use an initial ramdisk and build
> everything as modules.  Issue here is OpenWRT doesn't currently have the
> scripting/tools for runtime probing of a storage subsystem.  I think this
> is a fine approach if the committers are up for all the change this will
> entail.


That's the approach on devices with a serial port or video output where you can see why a box hung on boot and can diagnose and fix it.  That's not the approach on any device where it has to boot reliably.

Too much effort/risk for little benefit.


> Alternatively x86 could be broken into more builds.  This would emulate
> how OpenWRT works for all other devices.  "generic" would likely be 2-3
> distinct builds.  "64" would be 4-6 distinct builds.  Issue here is how
> many distinct builds should be created?


Still not convinced.  Show many an x86 platform that is nearly as constrained as an MT7621 SoC, and then I'll consider this...


> If one was to go this direction, I suppose there might be "giant" or
> "desktop" build.  Each hypervisor could use a target, include "hardware"
> guaranteed to be present.  Then build all network drivers as modules (so
> any device can be passed-in).
> 


The number of interfaces supported by virtualization (at least in KVM) are quite limited (e1000/igbe, ixgbe, rtl8139, and virtio) so I don't see this as much of a problem.



> Examples of things which don't seem to fit well are CONFIG_AGP and
> CONFIG_DRM.  I would expect those for a desktop Linux distribution due
> to GUI.  For OpenWRT which tends towards networking/embedded those seem
> out of place.  CONFIG_FB is similar, though some devices do have actual
> limited frame-buffers.


There are many mini-ITX boards suitable for networking (based on ATOM or Via C7 chips) that have VGA hardware and are adequately supported by the FB drivers doing console emulation...  Especially since during prototyping, a crash or panic might have time to write helpful messages to video memory, but not be able to write data out a relatively slow serial port before everything goes sideways.



> Another item is the goal of trying to self-host.  Being able to self-host
> is a worthy goal, but that has very distinct needs from an embedded
> networking device.


"Self-host"?  Meaning what in this case?  You want to build your images on the same box that you run your firewall?

-Philip