Functional octoprint won't connect on port 5000

Get Help Networking
1

What is the problem?
My octoprint instance has stopped accepting connections on port 5000 yet is successfully sending gcode to my printer. I've checked that the python3 process is running (it is) and is listening on TCP port 5000 (it is). The haproxy process gives the "The OctoPrint server is currently not running" message that it displays when it gets a 503 service unavailable from trying to connect.

Due to the fact that my printer is actively printing, I know that octoprint is functional. But octoprint is also not accepting connections on port 5000 to allow me to check status, see progress, etc.

What did you already try to solve it?

I found that I could successfully connect to port 80.

% nc -v 127.0.0.1 80
Connection 127.0.0.1 80 port [tcp/http] succeeded!

I found that I could not successfully connect to port 5000:

% nc -v 127.0.0.1 5000
[ just sits there, doing nothing ]

I've looked in the octoprint.log for clues. Nothing unusual jumps out at me.

With strace, I've explored the activities of the python3 process and its threads. I can see the actions of the thread that's writing gcode to the USB bus:

…
[pid  7542] clock_gettime(CLOCK_MONOTONIC, {tv_sec=243218, tv_nsec=773710006}) = 0
[pid  7542] futex(0x433a9c, FUTEX_WAKE_PRIVATE, 1) = 1
[pid  7542] write(17, "N355900 G1 F900 X178.529 Y169.19"..., 49 <unfinished ...>
[pid  7542] <... write resumed> )       = 49
[pid  7542] gettimeofday({tv_sec=1615066509, tv_usec=414354}, NULL) = 0
[pid  7542] futex(0x433a98, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1615066509, tv_nsec=419354000}, FUTEX_BITSET_MATCH_ANY <unfinished ...>
[pid  7542] <... futex resumed> )       = 0
[pid  7542] futex(0x433aa0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
…

But this doesn't help me figure out why another thread isn't responding on port 5000.

I've explored the logs to see if I can determine the root cause of the problem. I don't want to restart the octoprint python3 process because I'm concerned it will stop sending gcode to my printer during my current (32-hour) print. I'm hesitant to restart haproxy for the same reason, though that will likely be my next step. I thought I'd reach out to the community before I take steps that could potentially kill my print mid-stream.

Logs (syslog, dmesg, ... no logs, no support)

Output of ps:

% ps -efl | egrep 'haproxy|python'
4 S pi         426     1  1  80   0 - 225973 futex_ Mar03 ?       01:09:11 /home/pi/oprint/bin/python3 /home/pi/oprint/bin/octoprint serve --host=127.0.0.1 --port=5000
4 S root       530     1  0  80   0 -  3741 -      Mar03 ?        00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
5 S haproxy    539   530  0  80   0 -  4075 -      Mar03 ?        00:01:30 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid

Output of netstat:

% sudo netstat -tulpen | grep LISTEN
tcp      129      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      1000       20034      426/python3
tcp        0      0 192.168.4.40:8080       0.0.0.0:*               LISTEN      0          442570     9988/./mjpg_streame
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          17422      525/sshd
tcp6       0      0 :::80                   :::*                    LISTEN      0          15220      539/haproxy
tcp6       0      0 :::22                   :::*                    LISTEN      0          17424      525/sshd
tcp6       0      0 :::443                  :::*                    LISTEN      0          15221      539/haproxy

I can get any strace log needed, but several seconds gave me over 10,000 lines. I don't want to include that here.

The end of octoprint.log:

% tail .octoprint/logs/octoprint.log
2021-03-06 16:29:04,661 - octoprint.plugins.tplinksmartplug - INFO - Object
2021-03-06 16:29:04,662 - octoprint.plugins.tplinksmartplug - INFO - MoeSizzlac_-_Mandalorian_Chest_Armor_2.stl
2021-03-06 16:31:35,136 - octoprint.server.heartbeat - INFO - Server heartbeat <3
2021-03-06 16:31:42,447 - octoprint.plugins.tracking - INFO - Sent tracking event ping, payload: {'octoprint_uptime': 242994}
2021-03-06 16:37:51,636 - octoprint.plugins.tplinksmartplug - INFO - Object
2021-03-06 16:37:51,637 - octoprint.plugins.tplinksmartplug - INFO - NONMESH
2021-03-06 16:43:01,694 - octoprint.plugins.tplinksmartplug - INFO - Object
2021-03-06 16:43:01,696 - octoprint.plugins.tplinksmartplug - INFO - MoeSizzlac_-_Mandalorian_Chest_Armor_2.stl
2021-03-06 16:46:35,138 - octoprint.server.heartbeat - INFO - Server heartbeat <3
2021-03-06 16:46:42,323 - octoprint.plugins.tracking - INFO - Sent tracking event ping, payload: {'octoprint_uptime': 243894}

Additional information about your network (Hardware you are trying to connect to, hardware you are trying to connect from, router, access point, used operating systems, ...)

Everything about the node otherwise is fine. I can ssh to the raspberry pi, which is 192.168.4.40 on my local network. I can connect to haproxy.

2

Have you checked the browser settings:

3

I’ve left everything at its default of having haproxy listen on port 80 and OctoPrint listening on port 5000. Before the print, I could connect with any browser. (It’s how I started the print, after all.) So I don’t think browser settings are going to affect this inability to establish a connection on port 5000.

If it helps, I have tried connecting to 5000 with Chrome, curl, netcat (nc), and OctoPod. I’ve tried connecting from five different nodes, including the raspberry pi itself.

4

OctoPi doesn't expose port 5000 by default to any devices other than localhost/loopback address (127.0.0.1). So you have to go through haproxy to access it. If haproxy is not getting a connection, something about the server is broken - this may only be the web server side of it, the printing side can carry on if the server crashes in some situations.

Try restarting it (when the print finishes) or if you want to bypass haproxy, you have to be on the device itself - SSH and curl 127.0.0.1:5000 or something. If none of this works, it is more likely that something in the server has crashed - and you will have to restart.

1 Like
5

Thanks for the info, Charlie. After several more hours of printing, the print eventually failed. Python crashed with a "MemoryError". This Raspberry Pi has 4 GB of RAM, which is similar to what I've used before, so I'm not sure what's going on with this server.

6

Yeah, something's definitely up there. Could you grab the full octoprint.log and upload it? Maybe there are some details about what went wrong.

7

The log recommended that I file a bug with OctoPrint's bug tracker, so I did. The full octoprint.log is there. MemoryError during a print · Issue #4042 · OctoPrint/OctoPrint · GitHub

8

is there anything else running on the Pi besides octoprint? also, you didn't put in what versions of octoprint etc the pi is running. a lot of variables that cannot be answered without all the information requested