Restoring (old) config backups and

[Luiz Angelo Daros de Luca]

Hello,

> The problems we've observed with the backup *restore* functionality are:
>
> * It is applied with a simple reboot, and _apparently_, it has no hooks
> or other way to ensure system sanity before (or after) the backup restore.
>
> Which would be fine if the following point was never true:
>
> * End users are likely to use the backup-restore facility across
> sysupgrade (i.e. use older backups).
>
>
>
> Some possible root causes of the issues created on backup restoration:
>
> 1. /etc/uci-defaults is not "reset" like a sysupgrade would.  When the
> backup is "outdated" from the PoV of current packages in the running
> firmware image, it can cause issues ranging from minor, to severe.

It would be interesting to have a way to do firstboot+restore in a
single step (as cmdline can sysupgrade/restore).
However uci-defaults seems to be used to generate missing
configuration, not converting an existing one. A package
update could (ab)use this to migrate an existing configuration, but it
does seem to be the ideal usage (there are pre/post scripts).

> 2. Account information is not properly sanitized/recreated (/etc/passwd,
> shadow, group).  Thus, system accounts that were meant to be used by
> packages might be missing or have incorrect contents.   This is a
> consequence of (1), thus not a "root cause", but I find it relevant
> enough to mention it explicitly.
>
> If things fail to start because of missing system users, it is bad.
> When they *don't* fail to start and instead run as root, it is arguably
> *worse*.

It is a harder problem as it is not a simple migration but a merge process.
Somehow, a merge process will have to read both files at the same time
and merge them.
Today, the backup is restored directly over existing files. The merge
process would require
to expand it to a temporary directory and do the merge process.

> Some possible solutions:
>
> * It is not too hard to bake a firmware image uuid, randomly generated
> at build-time, add it to the backup tarball, and use it to detect the
> fact that we (re-)booted into a new environment that should go through
> /etc/uci-defaults processing as if it had been through a sysupgrade.

I guess uci-defaults is not ideal for that task. Those scripts are
meant to run once and get discarded after that.
UUID gives less info than BUILD_ID. It would be nice to have some form
of version information inside the backup,
even if it is informational only, like a /etc/.config_restored_version.

> This requires /etc/uci-defaults scripts that can handle being run when
> the changes they need to apply are already in place, which is *already*
> a best-practice, but still...

I imagine it could be possible to have something similar to database
migration scripts, where a set of atomic migrations occurs between
version x and version x+1. They would run only when
/etc/.config_restored_version exists. They should run immediately
after the restore happens. After that, /etc/.config_restored_version
could be removed or, in case a migration script fails, identify the
last working step.
Ideally, all these processes should be executed in a form of
"snapshot" of existing rootfs (a new layer of overlayfs?) and just
applied when it is finished.

It looks better than an "option migrated_featureX 1".

> Alternatively, or perhaps additionally to uci-defaults, it could run
> package-provided scripts from a directory on restore (pre-reboot,
> after-reboot).

It would be better if all migration could occur immediately after the
restore. However, there are two moments when a restore happens: when
the user asks for it and during
a sysupgrade that keeps existing configuration or provides a backup file.

Regards,

---
     Luiz Angelo Daros de Luca
            luizluca at gmail.com