Python3 upgrade fails

What is the problem?

failed to install python3-dev

What did you already try to solve it?

I log into my Pi via SSH

I prefer to use the screen command
Then

curl -L https://get.octoprint.org/py3/upgrade.py --output upgrade.py
python3 upgrade.py

ok so far

Checking package list for python3-dev
Installing python3-dev...
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
ERROR: failed to install python3-dev
Please try manually

Cleaning up...
Fatal error: Exiting

So i do sudo dpkg --configure -a
It asking me something a new version of config file of openssh. because i dont use it i choose the maintainer's version.

Errors were encountered while processing:
 libmagic1:armhf
 file

Then i did sudo dpkg --configure -a again

pi@octopi:~ $ sudo dpkg --configure -a
dpkg: dependency problems prevent configuration of libmagic1:armhf:
 libmagic1:armhf depends on libmagic-mgc (= 1:5.35-4+deb10u2); however:
  Package libmagic-mgc is not installed.

dpkg: error processing package libmagic1:armhf (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of file:
 file depends on libmagic1 (= 1:5.35-4+deb10u2); however:
  Package libmagic1:armhf is not configured yet.

dpkg: error processing package file (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 libmagic1:armhf
 file

Then again

curl -L https://get.octoprint.org/py3/upgrade.py --output upgrade.py
python3 upgrade.py

Then i got a lot of Cache entry deserialization failed, entry ignored messages.
last line is Installing collected packages and then the connection get lost.

Have you tried running in safe mode?

no

Did running in safe mode solve the problem?

donno

Systeminfo Bundle

You can download this in OctoPrint's System Information dialog ... no bundle, no support!)
octoprint-systeminfo-20211021125017.zip (3.5 KB)

Additional information about your setup

OctoPrint version, OctoPi version, printer, firmware, browser, operating system, ... as much data as possible
OctoPrint Version 1.7.0
OctoPi Version 0.17.0, running on Raspberry Pi 3 Model B Plus Rev 1.3
Win10
Firefox 93.0
Marlin

You have a throttle state:

env.plugins.pi_support.throttle_state: 0x80000

You may cool the Pi a bit more.

Hhhmmm

pi@octopi:~ $ /opt/vc/bin/vcgencmd get_throttled
throttled=0x80000

0x80000 Soft temperature limit has occurred (Source)

pi@octopi:~ $ vcgencmd measure_temp
temp=34.3'C

It occurred at some point previously, not necessarily currently - that would be 0x80008

Anyway, back to the Python 3 upgrade.

Can you still get into OctoPrint? Can you get into the Pi at all via SSH?

Cache entry deserialization should not make a difference to the operation of the script, this is something internal to pip and it does manage to install successfully anyway.

If it was installing, then it was most of the way there - it's good you managed to get past the dpkg problem as well, once you get to installation most of the hard work is done.

I rebooted the Pi.
Still have env.plugins.pi_support.throttle_state : 0x80000
Then sudo dpkg --configure -a
sudo reboot
then

curl -L https://get.octoprint.org/py3/upgrade.py --output upgrade.py
python3 upgrade.py

E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
ERROR: failed to install python3-dev

So I found this page: apt - Can't install or upgrade any package - Ask Ubuntu
and tried

sudo apt-get update --fix-missing
sudo dpkg --configure -a
sudo apt-get install -f 

Successfully installed python3-dev
Seems the sudo apt-get install -f solved it.

Edit: it still fail

Nope, when I lost the connection, it only helps to unplug the power of the pi.

OK still fail

in the end I got the tons of cache entry message. The last line says Installing collected packages and then the connection get lost.
I reconnected and switched to the screen (screen -r)

 call
    raise attempt.get()
  File "/home/pi/oprint/share/python-wheels/retrying-1.3.3-py2.py3-none-any.whl/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/home/pi/oprint/share/python-wheels/six-1.12.0-py2.py3-none-any.whl/six.py", line 693, in reraise
    raise value
  File "/home/pi/oprint/share/python-wheels/retrying-1.3.3-py2.py3-none-any.whl/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 111, in rmtree
    onerror=rmtree_errorhandler)
  File "/usr/lib/python3.7/shutil.py", line 487, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/usr/lib/python3.7/shutil.py", line 485, in rmtree
    fd = os.open(path, os.O_RDONLY)
PermissionError: [Errno 13] Permission denied: '/tmp/pip-req-tracker-wvgu_wok'
ERROR: OctoPrint failed to install

Because somewhere in the log I red:
Package libmagic-mgc is not installed
I did

sudo apt-get update
sudo apt-get install libmagic-mgc

still no luck

This is strange - is this a stock OctoPi install? Have you done any customizations/installed anything else besides OctoPrint/plugins?

The script runs sudo apt-get install python3-dev at the step that failed initially, but then it got further the second try, failing with something within pip. I don't really know what's broken on your install at the moment, but will keep thinking.