Will this happen again?

This version of the Python environment is not supported for direct updates.

To reduce the likelihood of running into update problems in outdated environments, the following minimal versions are required to use this updater to update OctoPrint or any installed plugins:

I installed OctoPi on a Raspberry Pi 3B+ a few years ago and have been updating it's software and plugins religiously.

Now I get the above message and information that tells me I have to backup my setting and basically rebuild OctoPi with a new install and reinstall all the Plugins, etc. and them bury it back inside the printer.

So, a few years down the road from now, is this going to rise up and bite me again?

Very likely once the python 2 support is eliminated and everything is migrated to Python 3. However, the backup/restore process of all the plugins, etc. is very easy to do with the Backup & Restore plugin now bundled with OctoPrint. The only ones it won't catch are those that aren't published to the Plugin Repository, but the settings will transfer.

I guess I don't understand why upgrading Python wipes out OctoPrint. Why are they not separated?

Python 2 versus Python 3: It's Different This Time
Python 2 Countdown Clock

I will wait until Python 3 for updating any of my OctoPi instances.
Sorry, but I think this is silly having to wipe and reinstall everything just to upgrade OctoPi.
OctoPi considers itself an 'OS' of sorts (just read its Into page), but its the only OS I know of that requires a wipe and reinstall to upgrade.

It doesn't, but upgrading it properly so that everything works afterwards - as with every Raspbian flavor - is more hassle than simply reflashing. And upgrade attempts by users who are not very familiar with Linux environments cause a ton of support overhead as well, making the reflash route more attractive for all involved parties.

If you have ideas how to improve the situation in a way that it does scale and allows to keep the underlying image updated and running reliably without user interaction - always happy for contributions.

In any case, I strongly suggest to always keep any kind of computer (which a Pi is) accessible enough to be able to physically reach it for maintenance. An SD card can also just die. A power regulator can burn out. A USB cable can break.

This right here. Fresh install takes MUCH less testing than an upgrade. Upgrades need to account for any changes that might have been made, along with many other factors.