Forum offline for repair today (9-Feb)

[B]

With all due respect, your organization needs an outside party to come 
in, do a postmortem, review your processes, and make improvements.

I feel like stronger language would be appropriate here, but I'm going 
to leave it at that for now.



On 2/9/25 7:07 AM, Ted Hess wrote:
> Hi all -
>
> Last nite I was doing a routine forum upgrade and ran into a number of 
> problems caused by things like a policy change at the docker.io image 
> repository access for free accounts, a failed database service upgrade 
> (postgres v13 to v15) and now a failed volume mount at Digital Ocean 
> during the upgrade. The first 2 items were worked around OK but the 
> volume mount problem has caused us to lose all uploaded data like 
> posted images and forum decorations and customizations.
>
> There is a possibility of recovering our lost data from DO which I am 
> working on currently. In the meantime, the forum is offline while we 
> try to retrieve the data. The most painful part of this process is 
> that the forum backups were on the missing volume. It is possible that 
> the virtual block volume mounted is not the correct one and there is a 
> (remote) possibility of getting the correct one back. In the meantime, 
> the forum will remain down until we find a resolution. In the worse 
> scenario, we just lost all our uploaded data AND older backups - not 
> good.
>
> /ted
>
>
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel