[OpenWrt-Devel] Progress on Reproducible Builds

[bnewbold at robocracy.org]

Hello!

I have been making some progress towards reproducible (aka, deterministic) 
builds of OpenWRT packages and "images" (target artifacts). Some emails 
from earlier this year on the topic were:

   https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033667.html
   https://www.mail-archive.com/openwrt-devel@lists.openwrt.org/msg32534.html

I have some patches to the OpenWRT build system and a small number of core 
packages (host and target), and am writing now to ask how best to start 
submitting them. I could:

1. Submit all patches now as-is (following OpenWRT conventions)
2. Wait until the work reaches a milestone (eg, most image artifacts are
    reproducible) and submit then
3. Upstream patches to other projects first (busybox and squashfs-tools)
4. Or something else

The patches can be browsed online here:

   https://github.com/bnewbold/openwrt-repro/compare/master...repro

There are some changes to OpenWRT's build makefiles and scripts; patches 
to host tools; and patches to core target packages.

Current progress can be checked between build results from these patches:

   http://repro.bnewbold.the-nsa.org/openwrt-results/

vs. those generated by the debian reproducible builds project directly 
from openwrt.git:

   https://reproducible.debian.net/openwrt/openwrt.html

The oprofile and gdb packages don't reproduce due to simple __DATE__ 
timestamps; I assume these don't get built in to most release images so I 
haven't prioritized them. There are still issues with squashfs generation 
and I haven't touched ext4 filesystem generation yet. Also, I assume more 
issues will crop up if changes to building username/uid, hostname, and 
(UTC) date are incorporated into the build process.

I think a valuable milestone will be the ability for independent parties 
to reproduce all the .ipkg, kernel, rootfs, and image files for a given 
architecture, given the release tag (for the openwrt repo and any package 
feeds) and the "config.diff" file included in releases on 
http://downloads.openwrt.org/. It would be helpful if exact source 
checksums (eg, git commits if git was used) and perhaps basic build system 
metadata (eg, build system architecture, host GCC version numbers) were 
somehow included in a metadata file in the release directories. I'm unsure 
how helpful or important it would be to have the SDK or Image Builders 
either be reproducible themselves or to generate reproducible artifacts.

--bryan
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel