sysupgrade broken on imx nand targets (and maybe others too)

[Koen Vandeputte]

On Tue, Jan 24, 2023 at 11:43 PM Lanchon <lanchon at gmail.com> wrote:
>
>
>
> On 1/24/23 18:08, Koen Vandeputte wrote:
> >
> > Hi,
> >
> > I think our previous mails overlapped a bit as  I didn't notice your
> > previous response :)
> >
> > I'll send the data tomorrow.
> > I'm also wondering if adding a sleep before ubiformat in the old way
> > would influence/break it's behaviour?
> possibly yes
> >
> >
> > Piotr,
> > Would you have any idea here?
> >
> >
> > Thanks again for your efforts,
> >
> > Koen
>
>
> if your active watchdog device is /dev/MyWatchdog, then...
>
>
> just before the line:
>
> ${gz}cat "$ubi_file" | ubiformat "/dev/mtd$mtdnum" -y -f - && ubiattach
> -m "$mtdnum"
>
> you could add this to try disable the watchdog:
>
> echo -n V >/dev/MyWatchdog
>
>
> or else you could tickle the watchdog while flashing like this:
>
> ${gz}cat "$ubi_file" | ubiformat "/dev/mtd$mtdnum" -y -f - 2>&1 | tee
> /dev/MyWatchdog && ubiattach -m "$mtdnum"
>
> BUT... this is a hack, cause shell option pipefail would be needed to
> detect a failure of ubiformat then, as normally only the exit code of
> the last piped process gets returned from the pipe expression.
>
>
> so these are just tests, not fixes.
>
>
> another dirty thing you could do is doubling the watchdog timeout period
> for your platform. this at least could maybe be accepted as a stopgap
> band aid in the openwrt tree. but the race condition remains, and it
> will bite back sometime. a real fix for the race should be implemented.
>
> BTW your watchdog seems to be:
> https://www.kernelconfig.io/config_imx2_wdt
>
>
> thanks!
>
>


Hi all,

Hardware:
Gateworks Ventana gw5200, naked without any additional hardware attached.

Full normal bootlog:
https://pastebin.com/raw/nSBnrHxN

Watchdog info:
root at OpenWrt:~# ubus call system watchdog
{
"status": "running",
"timeout": 30,
"frequency": 5,
"magicclose": false
}


Normal reboot command issued as user after boot:
--> a reboot on Gateworks Ventana is handled by waking the watchdog
which triggers a GPIO, controlling a PMIC to reset power

[   56.791836] br-wan: port 1(eth0) entered disabled state
[   56.801236] device eth0 left promiscuous mode
[   56.805828] br-wan: port 1(eth0) entered disabled state
* reboot command issued by me here *
crond[1416]: USER root pid 2191 cmd /usr/sbin/logrotate
/etc/logrotate/logrotate.conf > /var/log/logrotate.log
[   61.139944] ci_hdrc ci_hdrc.1: remove, state 1
[   61.144418] usb usb2: USB disconnect, device number 1
[   61.150316] ci_hdrc ci_hdrc.1: USB bus 2 deregistered
[   61.155792] ci_hdrc ci_hdrc.0: remove, state 1
[   61.160257] usb usb1: USB disconnect, device number 1
[   61.166336] ci_hdrc ci_hdrc.0: USB bus 1 deregistered
[   61.173094] imx2-wdt 20bc000.watchdog: Device shutdown: Expect reboot!
[   61.179787] reboot: Restarting system


Full log sysupgrade failing:  (cat ubifile | ubiformat -f -)
https://pastebin.com/raw/QPNbe49K
Time from sysupgrade-start to reboot --> 23.38 sec

Full log sysupgrade working:  (ubiformat -f /tmp/ubifile)
https://pastebin.com/raw/qtSMnbDh
Time from sysupgrade-start to reboot --> 33.55 sec



Again ..
I'm NOT saying that feeding the ubi directly is a proper fix.
I'm saying that the watchdog itself seems NOT to be the cause.
It is used as a way to powercycle the board when a "reboot" command is issued.

According to me, somewhere during upgrade a 'reboot' command gets
triggered somewhere at a wrong time ..

Kind regards,

Koen