Ocotprint Web Interface - 500 error

What is the problem?

The Octotprint webpage is running into a 500 error after a while. And I get Websocket issues in the console log of the server. OctoApp is getting data regardless of an active 500 http error on the octoprint webpage. The webpage itself does so to, a page refresh will let me know if I have run into a 500 http error. Logging isn't saying me a lot to me on this issue.

What did you already try to solve it?

Shut down plugins. Read various post. Tried to configure websockets in my reverse proxy (I use the reverse proxy of my Synology NAS). Disconnected all USB camera's from my Pi4-2Gb bar the PiCam itself.

Have you tried running in safe mode?

Yes. This gave less 500 errors. But not none.

Did running in safe mode solve the problem?

No really. It did get better, but that good be dumb luck. It happens randomly.

Systeminfo Bundle

octoprint-systeminfo-20250119204419.zip (613.6 KB)

Additional information about your setup

Octoprint 1.10.3 on a Pi4-2Gb running from an SSD drive connected through USB3.0. PSU is a Raspberry PI 3A PSU.

Regarding the Websocket issue which might be related. I don't know. I get this error in console.log;

wss://URL/sockjs/715/vepy1k15/websocket' failed: n.exports @ packed_libs.js?409a9b49

Where can I find websocket configuration documentation?

I there :slight_smile:

First of all just fyi - OctoPi already comes with a reverse proxy.

Since OctoApp still works we can probably rule out network issues. Have you tried to run it without a second reverse proxy?

Tagging @foosel since I can't answer you websockets question - all I can send you is the link to the REST api documentation REST API — OctoPrint master documentation

Morning! Thanks for your reply. I read somewhere on the forum, can not remember where, that it was not adviced to run the reverse proxy on the same device. Also, I use the reverse proxy for several applications (non-octoprint related) and from efficiency point of view, would like to have just the one reverse proxy.

However, I do use HAProxy on the octoprint instance. This to route traffic from outside to the attached webcams (which are currently not running as a for mentioned issue keeps happening).

What is your advice in my senario?

For completeness I add my reverse proxy test overview from octoprint.

Oh! And what logging would be best to activate to get more information for this issue? All logging I have seen up to now doesn't give me error messages. At most I get a warning message that I think might be related. Like;

2025-01-20 04:38:52,940 - octoprint.server.views - DEBUG - Path / not yet cached (key: ui:_default:http://192.168.2.12/:en), signaling as missing
2025-01-20 08:47:22,189 - octoprint.server.heartbeat - INFO - Server heartbeat <3

in octoprint.log

Thanks in advance for your insights!

If you throw another reverse proxy in front of OctoPrint, you'll also need to configure that as a trusted downstream:

That's also the guide that you should cross check regarding the general configuration of any additional reverse proxies you throw in front of OctoPrint.

With regards to your web socket issues, I see a bunch of authentication errors on the socket:

2025-01-19 20:19:33,069 - octoprint.server.util.sockjs - WARNING - Unknown user/session combo: ruud:2b1f4288326a1a50edf8b3c974a63c478b5f724ff746c532c0fd5def4f2343ab

That sounds like something is getting out of sync somewhere, but I'm unclear on whether that's triggered by OctoApp or something else.

The Internal Server errors sound weird and don't reflect in the log, which makes me wonder if they are actually being generated by OctoPrint and not rather in your additional reverse proxy. Try accessing OctoPrint directly for a bit and see if that solves it.

Hi @foosel,

thank you for your time and response. I have configured host-headers within my Proxy Server that is in front of Octoprint. These are the headers configured;

It is running with Proxy HTTP-version 1.1

Is this configuration correct for Octoprint or am I missing something?

I notice my Reverse Proxy Test is giving IP of my (synology) Reverse Proxy as the client IP. Is this an indicator something is off? If so, what could I change to improve the situation?

Best regards,

Ruud

As I said:

If you throw another reverse proxy in front of OctoPrint, you'll also need to configure that as a trusted downstream:

That not having been done is the reason the client IP is wrong.

IIRC Synology's proxy config correctly, there's an option somewhere under advanced configuration that allows you to enable websocket support.

I don't know if it properly replaces the variables that you've entered into the form there, so I can't say if that will work or not. I run a Synology NAS myself, but their reverse proxy configuration was too much of a black box for me and I've since switched to having my services run behind Traefik in a Docker container.

Hi @foosel,

thanks for your response. Forgot added the HAproxy from the octoprint instance as a trusted proxy to the synology settings. With that done;

I do get the client IP address correctly on the local network. From outside the reverse proxy tester still gives the synology reverse proxy as client IP. I will investigate a little further.

Switching to Traefik as main reverse proxy feels like a move that I'll have to make.

Thanks for you time, effort, thoughts and suggestions thus far!

Regards,

Ruud

You have to tell OctoPrint about the trusted proxy, as described in the linked post.

I, missed that detail.... Sorry!

I just started a new print, after that I will make the change.

During the current print, upon switching from one connecting USB webcam stream to the other USB webcam stream (both comming from a C270 camera), I received an 500 error whilest logged in locally (not via proxy, but direct). Also ssh is not available anymore. Printing process is still going and the PiCamera (CSI) is streaming without issues. OctoApp is display telemetry from the printer in realtime (temp etc) thourgh the internet (thus, through the reserve proxy).

Is it a fair assumtion that the issue is related to the USB webcam streaming? My third USB camera, endoscope camera, is also not available (was before the 500 error occurred).

I will now wait 2 hours for the print to finish.

regards,

Ruud

If you open the Browserconsole (F12 on the keyboard) and switch to the Network Tab, what does it show?

It just stats the 500 error.

Hm... with nothing in the logs that's a tricky one. I find it somewhat unlikely that the webcam could cause the OctoPrint server to throw an HTTP 500, and I also find it a bit odd that the problem reduced but didn't go away when you went into safe mode.

I also can't find a single call to the web interface in the haproxy.log in your system info bundle that reflects an HTTP 500, so something is definitely being weird here.

The fact that it's also no longer reachable via SSH is an additional thing that doesn't make a ton of sense here. So maybe, just maybe, it's related to the webcam after all, and that's somehow causing the Pi to go crazy, maybe by eating up too many resources. I'm honestly out of ideas right now of what else to try, apart from running the server from an interactive SSH session and try to see if you can reproduce the error there and maybe see something logged to the default output that for some reason never makes it into the log files, but since you said it's happening randomly, that might not even be an option.

Hi @foosel,

thanks for your honest response. I will try logging with SSH as well.

And setting up camera-streamer on a standalone raspberry pi.

Regards,

Ruud

1 Like

Short update;

I have added the downstream proxy to octoprint via config.yaml. External IP address of client is now shown in the reverse proxy tester.

Websocket conenction still gives the same error.

When I have updates I will share them here.

Regards,

Ruud

And another update. Whilest printing I receive a message that octoprint is not able to resolve octoprint.org. I checked in the settings under the connectivity tab and 1.1.1.1 could be resolved. octoprint.org or other domain names could not.

All 3 USB camera's couldn't be resolved. At the start of the print they could. The Pi Camera (CSI) still gives a crips live images.

As the printjob was almost finished I sliced the next printjob and wanted to upload it, by drag and drop. Webpage reacted with the overlag for drag'n'drop options per screen half. Dropped the gcode, but nothing was being uploaded. Hit F5 and hit another 500 error. On the local network and from outside.

As soon as I have the time I will transfer the camera's to there own camera-streamer instance. I will report back with the results. Until then, I'll try to live with this situation.

Up untill this point, thanks for all the help!!!

Regards,

Ruud

1 Like

Hi all,

Currently doing a longer print (18h) with octoprint in Safe Mode. Gui has thrown a 500 error. Interestingly enough the PiCamera is streaming, in various formats, from port 8080.

So it does not seem to be a USB overload issue. As no usb camera's are connected. The powered USB hub I use is also disconnected.

Is this the moment (when the print has finished) to do a fresh install of Octoprint/OctoPi?

I see no logical reasons for this 500 error left, apart from a curious mal configuration of Octoprint.

Or does someone have an idea what else I could try?

Regards,

Ruud

Is your DNS and eventually DHCP set up correctly? 1.1.1.1 is already an IP address that needs no resolver, but octoprint.org does. If that is not working you should try to ping the hosts inside your local network and if that works, ping hosts outside of your local network, on the internet. If that does not work, check the DNS/DHCP setup in your LAN and/or WiFi network.

Thanks! I will check this. This was the first time I got an resolving issue. Whilst the pin was available through the local network. I will do ping tests when the current print has finished.

Another update.

I connected a display to the pi to see what was on the console during an 500 error.

It is a lot of errors related to ext4 access to the SSD storage;

[31415.380993] EXT4-fs error (device sda2) ext4_find_entry: 1664: inode #anumber: comm systemd: reading directory lblock 0
[31415.397572] EXT4-fs error (device sda2) ext4_find_entry: 1664: inode #anumber: comm cron: reading directory lblock 0
[31415.406280] EXT4-fs error (device sda2) ext4_find_entry: 1664: inode #anumber: comm systemd-udevd: reading directory lblock

And so on. Alternating with varying int's for #anumber

I did some google'n on EXT4-fs errors and found this post on another forum;

From which I drew the conclusion that it is either a power issue resulting in diskread errors. Or a faulty SSD resulting is diskread errors.

The weird thing (#1) is that is happens after a while during printing. Which makes me believe it is firstly a power issue. I will try a different PSU (I have a second Raspberry Pi 3amp PSU) and see if that keeps the error away.

The weird thing (#2) is that octoapp and the camera stream are still live in realtime. See below.

Untill the next time!

Regards,

Ruud

You can also try these commands to test the DNS:

nslookup (name of a host in your LAN/WiFi network, not an ip address)
nslookup octoprint.org
nslookup google.com

This should give you an ip address for the name you looked up as well as the ip address of the server that resolved your query, looking something like this:

walter@zarquon ~/devel/linux $ nslookup octoprint.org
Server:         192.168.95.2
Address:        192.168.95.2#53

Non-authoritative answer:
Name:   octoprint.org
Address: 159.69.6.198
Name:   octoprint.org
Address: 2a01:4f8:1c0c:6958::1

1 Like