[OpenWrt-Devel] Preserving configs over sysupgrade on ath79/tiny broken for some boards (on 19.07 and master), possible solutions

[Paul Fertser]

Hi,

Status quo summary:

Many ath79/tiny devices will lose configs (almost) each time
sysupgrading. This is easy to fix for most of the boards that
consistently have compatible flash memory ICs (some TP-Links are not
elements of this set). The only easy fix for all the boards is to
revert the 4K sectors patch.


Rationale for using 4K erase sectors:

OpenWrt uses jffs2 filesystem for storing ssh host key, config files
and other auxilary information across reboots. For that to work at
least 5 erase blocks are needed. With regular 64 KiB sectors that
means your rootfs_data partition needs to be at least 320 KiB
big. This is clearly much more than enough, and you'll have to
sacrifice some of the features when packing your rootfs. Some flash
memory chips allow using 4 KiB erase sectors so the minimal
rootfs_data partition size becomes 20 KiB (in theory).

An example of why I think that is useful: I built a regular image
(sans PPP and LuCI) for my 4 MiB ath79 device with dnsmasq DNSSEC, SQM,
6in4 and udpxy. The whole firmware partition is 3776 KiB, of which
1442350 bytes go to kernel, 2424274 to rootfs and 116 KiB to
rootfs_data. With regular configuration 40 KiB of it are used, and 76
are free. And the system still has opkg so that I can easily install
tcpdump-mini or other diagnostic utilities to /tmp. This wouldn't be
possible (at least not with SQM) if I was using 64 KiB erase blocks on
this device.


Sysupgrade problem description:

During the upgrade process the saved configuration files are written
to the flash starting from the erase marker added to the sysupgrade
image during the pad-rootfs build image stage. If there is a mismatch
between BLOCKSIZE specified in the Makefile for the specific board and
erase block size as determined by the kernel by reading the flash
memory JEDEC ID and checking its capabilities from a table in spi-nor
driver then the erase marker will likely be misaligned. I've compiled
the list of all the ath79/tiny devices so that proper BLOCKSIZE could
be specified but faced an issue with some TP-Link boards that
according to certain sources can have factory memory with different
capabilities. The full list is at the end of this mail.


Possible solutions that come to mind:

1. Revert to using 64 KiB erase sectors on all ath79/tiny devices;

2. Fix BLOCKSIZE for those boards that are consistent, document
possible config loss for the others;

3. Change sysupgrade in a way to either:

3.1. When seeing the erase marker automatically align it to the
nearest erase sector boundary and act accordingly;

3.2. Ignore erase markers not placed at the runtime-detected erase
sector boundaries; make build system emit those at both 4K and 64K
locations;

3.3. Port kernel squashfs splitter logic to sysupgrade to make it find
the right place for the erase marker (or jffs2 with configs) fully on
its own.


That's how I see it, please fix me where I am wrong.


List of the ath79/tiny boards with references:

Buffalo WHR-G301N
http://openwrtwlan.blogspot.com/2012/11/wzr-g301ntftp.html
mx25l3205d
SECT_4K

DIR-615 E4
https://forum.archive.openwrt.org/viewtopic.php?id=52823
mx25l3205d
SECT_4K

PQI Air-Pen
http://www.srchack.org/article.php?story=20180924234149309
s25fl064k 8 MiB
(not important since 4K sectors not used on 8 MiB)

Netgear WNR612 v2
https://openwrt.org/toh/netgear/wnr612v2
mx25l3205d
SECT_4K

On Networks N150R (rebranded Netgear WNR612v2)
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=b6e1cbd8dd5b8fe3833547480abdbd57edb3238c
mx25l3205d
SECT_4K

Netgear WNR1000 v2
https://gist.github.com/SaltwaterC/ef49c8a4df81c8896153
w25q32
SECT_4K

Netgear WNR2000 v3
https://oldwiki.archive.openwrt.org/toh/netgear/wnr2000
mx25l3206e (same JEDEC ID as mx25l3205d)
SECT_4K

TP-Link TL-MR10U
https://openwrt.org/toh/tp-link/tl-mr10u
Spansion FL032PIF 32Mbit or Eon EN25Q32B
64 KiB sectors

TP-Link TL-MR3020 v1
https://openwrt.org/toh/tp-link/tl-mr3020
S25FL032P
64 KiB sectors

TP-Link TL-MR3040 v2
https://openwrt.org/toh/tp-link/tl-mr3040
S25FL032P
64 KiB sectors

TP-Link TL-MR3220 v1
https://www.kaskus.co.id/show_post/58181950d9d77097618b4569/791/-
s25sl032p
64 KiB sectors

TP-Link TL-MR3420 v1
https://openwrt.org/toh/tp-link/tl-mr3420
v1: MX25L3206E; v1.1: F32-100HIP; v1.2: W25Q32BV/FL032PIF
v1: SECT_4K; v1.1: SECT_4K; v1.1: SECT_4/64 KiB

TP-Link TL-MR3420 v2
https://openwrt.org/toh/tp-link/tl-mr3420
v2.3 winbond or spansion?
???

TP-Link TL-WA901ND v2
https://openwrt.org/toh/tp-link/tl-wa901nd
s25sl032a
64 KiB sectors

TP-Link TL-WR703N
https://openwrt.org/toh/tp-link/tl-wr703n
s25sl032a
64 KiB sectors

TP-Link TL-WR740N v1/v2
https://openwrt.org/toh/tp-link/tl-wr740n
en25q32b
64 KiB sectors

TP-Link TL-WR740N v3
? https://eko.one.pl/forum/viewtopic.php?id=14257 ? en25f32 4k?
???
???

TP-Link TL-WR740N v4
http://bkil.blogspot.com/2014/12/tftp-secret-of-tl-wr740n-uncovered.html
s25sl032p
64 KiB sectors

TP-Link TL-WR741N/ND v1/v2
v1: ??? v2.4: https://openwrt.org/_media/media/tplink/tp-link-v2.4-serial-pins.jpg?cache= , 2.4: http://blog.hajma.cz/2014/02/tp-link-tl-wr741nd-unbricking.html
v1: ???; cFeon F32-100 en25f32, s25sl032p
???; SECT_4K, 64 KiB sectors

TP-Link TL-WR741N/ND v4
v4.3: https://openwrt.org/_media/media/tplink/tl-wr741/tl-wr741nd-v4_board.jpg?cache=
Spansion FL032P
64 KiB sectors

TP-Link TL-WR743ND v1
???
???
???

TP-Link TL-WR841N/ND v5/v6
???
???
???

TP-Link TL-WR841N/ND v7
https://openwrt.org/toh/tp-link/tl-wr841nd
Eon EN25F32
SECT_4K

TP-Link TL-WR841N/ND v8
https://openwrt.org/toh/tp-link/tl-wr841nd
Spansion FL032PIF
64 KiB sectors

TP-Link TL-WR841N/ND v9
https://openwrt.org/toh/tp-link/tl-wr841nd
Spansion FL032PIF
64 KiB sectors

TP-Link TL-WR841N/ND v10
https://openwrt.org/toh/tp-link/tl-wr841nd
Winbond 25Q32FVS1G
SECT_4K

TP-Link TL-WR841N/ND v11
https://openwrt.org/toh/tp-link/tl-wr841nd
Spansion FL032PIF
64 KiB sectors

TP-Link TL-WR841N/ND v12
https://openwrt.org/toh/tp-link/tl-wr841nd
Spansion FL032PIF
64 KiB sectors

TP-Link TL-WR940N v3
https://www.gargoyle-router.com/phpbb/viewtopic.php?t=8034
gd25q32
SECT_4K

TP-Link TL-WR940N v4
https://openwrt.org/_media/media/tplink/tl-wr940/tl-wr940n-v4_uart.jpg
gd25q32
SECT_4K

TP-Link TL-WR941ND v2/v3
https://openwrt.org/toh/tp-link/tl-wr941nd
the logs indicate that one of the v2.x devices mentioned has 8MB flash (prisma, RoundSparrow), while others had 4MB (juhosg (?), erreterr, sam.vanratt, _bbb_).
???

TP-Link TL-WR941ND v4
???
???
???

TP-Link TL-WR941ND v6
???
???
???

TP-Link TL-WR941ND v7 (CN)
???
???
???

-- 
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:fercerpav at gmail.com

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel