πŸ‡ΊπŸ‡¦ We stand with Ukraine! πŸ‡ΊπŸ‡¦

Octoprint Server will not start: Corrupted users.yaml

More of an FYI to help others and possible prevent it happening in the future. It took me about a week (20 mins daily here and there) of delving to find the problem and solve it. I will attempt to recreate it if I can...

What WAS the problem?

I will attempt to explain this the best I can.

Server running 1.8.1 on a Raspberry Pi 4b 2Gb, 32Gb SD

After installing plug-in updates to "OctoPrint-ExtraFileInfo-1.0.0" and "OctoPrint-PiSupport-2022.6.13", Octoprint service failed to restart, even rebooting Raspberry Pi failed to restart service.

What did you do to try to solve it?

ps -ef | grep -i octoprint | grep -i python Yielded Nothing
sudo service octoprint restart Yielded Nothing
Server would not start to allow the ability to download the System Info or Logs as Web interface would not load.
octoprint systeminfo Yielded nothing
octoprint systeminfo . Also yielded nothing

pi@octopi: ~/oprint/bin/octoprint

Starting the server via "octoprint" is deprecated, please use "octoprint serve" from now on.
2022-06-17 19:53:52,214 - octoprint.startup - INFO - *********************************************************************** *******
2022-06-17 19:53:52,216 - octoprint.startup - INFO - Starting OctoPrint 1.8.1
2022-06-17 19:53:52,217 - octoprint.startup - INFO - Starting in SAFE MODE. Third party plugins will be disabled!
2022-06-17 19:53:52,217 - octoprint.startup - INFO - Reason for safe mode: problem during last startup
2022-06-17 19:53:52,217 - octoprint.startup - INFO - *********************************************************************** *******
2022-06-17 19:53:52,481 - octoprint.util.connectivity.connectivity_checker - INFO...
...
2022-06-17 19:53:54,088 - octoprint.server - ERROR - Could not instantiate user manager octoprint.access.users.FilebasedUser Manager, falling back to FilebasedUserManager!
Traceback (most recent call last):
File "/home/pi/oprint/lib/python3.7/site-packages/octoprint/server/init.py", line 553, in run
userManager = clazz(groupManager)
File "/home/pi/oprint/lib/python3.7/site-packages/octoprint/access/users.py", line 508, in init
self._load()
File "/home/pi/oprint/lib/python3.7/site-packages/octoprint/access/users.py", line 512, in _load
data = yaml.load_from_file(path=self._userfile)
File "/home/pi/oprint/lib/python3.7/site-packages/octoprint/util/yaml.py", line 14, in load_from_file
return load_from_file(file=f)
File "/home/pi/oprint/lib/python3.7/site-packages/octoprint/util/yaml.py", line 28, in load_from_file
return yaml.load(file, Loader=SafeLoader)
File "/home/pi/oprint/lib/python3.7/site-packages/yaml/init.py", line 112, in load
loader = Loader(stream)
File "/home/pi/oprint/lib/python3.7/site-packages/yaml/loader.py", line 34, in init
Reader.init(self, stream)
File "/home/pi/oprint/lib/python3.7/site-packages/yaml/reader.py", line 85, in init
self.determine_encoding()
File "/home/pi/oprint/lib/python3.7/site-packages/yaml/reader.py", line 124, in determine_encoding
self.update_raw()
File "/home/pi/oprint/lib/python3.7/site-packages/yaml/reader.py", line 178, in update_raw
data = self.stream.read(size)
File "/home/pi/oprint/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
File "/home/pi/oprint/lib/python3.7/encodings/utf_8_sig.py", line 69, in _buffer_decode
return codecs.utf_8_decode(input, errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 1: invalid start byte

I figured it was a problem with the Python script for UTF-8, so I performed an APT-GET update and upgrade, this completed but did not fix the problem.

I did a downgrade to v 1.8.0 of Octoprint using:

pip install --force-reinstall OctoPrint==1.8.0 --no-cache-dir

Afterwards, restarting had no effect

It took me a bunch of Gooleing to find what "octoprint.access.users.FilebasedUser" referenced, the users.yaml file. I opened the file via SSH (not my preferred method)

pi@octopi:~/.octoprint $ nano users.yaml

the file appeared as either binary or possibly in some kind of character language like Chinese or Japanese? Since the file contains user PW data, it will not be "displayed" or uploaded.

on my Windows machine I extracted the users.yaml from an old backup and proceeded to rename the bad yaml file and SCP (pscp.exe) the backup to my pi:

pi@octopi:~/.octoprint $ mv users.yaml users.yaml.corrupt
C:\Users\...\Downloads>pscp  C:\temp\users.yaml.frombackup pi@octopi:.octoprint/users.yaml
pi@octopi's password:
users.yaml.frombackup     | 0 kB |   0.3 kB/s | ETA: 00:00:00 | 100%
sudo service octoprint restart

The service finally started and I was able to once again access the web interface of v1.8.0

Have you tried running in safe mode?

Yes via command line, unable to load server

pi@octopi:/home $ ~/oprint/bin/octoprint safemode
Safe mode flag set, OctoPrint will start in safe mode on next restart.

Did running in safe mode solve the problem?

No

Systeminfo Bundle

You can download this in OctoPrint's System Information dialog ... no bundle, no support!) FUNNY as No Service Start, No Download system info!

this is the v1.8.0 system info file, I'll upload my v1.8.1 file after re-upgrading.
octoprint-systeminfo-20220620182522.zip (673.7 KB)

Additional information about your setup

OctoPrint version, OctoPi version, printer, firmware, browser, operating system, ... as much data as possible

1.8.1 on a Raspberry Pi 4b 2Gb, 32Gb SD, downgraded to 1.8.0 during troubleshooting.
Kernel 5.10.103-v7l+
OctoPrint 1.8.0
Python 3.7.3
OctoPi 0.18.0
AnyCubic Mega Pro

Suggested change(s)?

Possibly remove the dependency on corrupted config file(s) for a "Super" Safe-Mode, to allow backup restoration.

oh and this is the only message I was getting via any browser connection, if you needed it:
https://octopi
or
https://octopi/recovery


The OctoPrint server is currently not running

If you just started up your Raspberry Pi, please wait a couple of seconds, then try to refresh this page.

If the issue persists, please log into your Raspberry Pi via SSH and check the following:

    Verify that the process is running: ps -ef | grep -i octoprint | grep -i python should show a python process:

    pi@octopi:~ $ ps -ef | grep -i octoprint | grep -i python
    pi 1441 1 6 11:12 ? 00:00:15 /home/pi/oprint/bin/python
    /home/pi/oprint/bin/octoprint --host=127.0.0.1 --port=5000

    If it isn't, the question is why. Take a look into ~/.octoprint/logs/octoprint.log, there might be an error logged in there that helps to determine what's wrong.
    You might also want to try if you can restart the server (if no obvious error is visible): sudo service octoprint restart.

If all that doesn't help to trouble shoot the issue, you can seek support on the OctoPrint Community Forum. Please provide your OctoPi and OctoPrint versions as well as your octoprint.log and explain what you already tried and observed as detailed as possible.

re-Upgrade to 1.8.1 seems to have completed without issue:
OctoPrint 1.8.1
Python 3.7.3
OctoPi 0.18.0
octoprint-systeminfo-20220622221408.zip (1.4 MB)

This does just seem like a case of file corruption. Sometimes it can happen if you unplug the RPi without shutting down. I've not seen corrupted config after an update, usually the actual update fails and you get a mixture of old and new, so that's new.

This is a tricky one - basically everything depends on starting up the user manager, to check if users are logged in to actually access stuff. For security, I think it is probably better that it fails to start, than allowing anyone (since authentication would be completely gone) to just take over the server with a backup.

My advice for the future, to save the time with stuff you don't understand - just ask a question on the forums or discord server and someone will try and help :slightly_smiling_face:

Im actually quite obsessive with properly shutting down *nix systems, barring power loss... it hasnt been improperly shutdown (that i am aware of) in months