Octoprint Process seems to get Killed in long running Print

What is the problem?
Octoprint stops after about 3 days of printing, in the syslog it seems that the kernel kill the process.

What did you already try to solve it?
Full reboot and clean but it happened again

Logs (octoprint.log, serial.log or output on terminal tab at a minimum, browser error console if UI issue ... no logs, no support!)syslog.log (15.8 KB)

Additional information about your setup (OctoPrint version, OctoPi version, printer, firmware, browser, operating system, ... as much data as possible)

I Am not so deep in the linux stuff, I found the log but have no idea why and how to solve, the octoprint log is about 50 MB big can upload it if requested.

And thanks for any help :slight_smile:

You should try this in Safe Mode to make sure that none of your third-party plugins are causing this.

If your log is 50MB then try to copy/paste here the last 200 lines of that, the part where it's going bad. Make sure to highlight that here then and press the button which looks like this in the editor: </>

1 Like

I have uninstalled all plugins after the first crash but it happened again here are the last 100 lines (to many characters for 200 and the log entrys are redundant)

2019-12-15 19:00:10,296 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:13,251 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2019-12-15 19:00:14,490 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:15,710 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:17,324 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:19,483 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2019-12-15 19:00:20,655 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:21,873 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:22,504 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:23,308 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:24,560 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:26,627 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2019-12-15 19:00:27,104 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:27,645 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:28,600 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:29,707 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError
2019-12-15 19:00:30,830 - octoprint.printer.standard - ERROR - Exception while adding communication log entry to callback <octoprint.server.util.sockjs.PrinterStateConnection object at 0x69ab93f0> connected to ::ffff:192.168.178.126
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/printer/standard.py", line 189, in _sendAddLogCallbacks
    callback.on_printer_add_log(data)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/server/util/sockjs.py", line 297, in on_printer_add_log
    self._logBacklog.append(data)
MemoryError

In the syslog in line 115 the kernel seems to kill the octoprint process, in the octoprint log file the last about 40000 lines are the same exception as seen in the pastel code above :confused:

Disk full or defect?

Octoprint said 11GB of free disk Space, and it happens After 3 days of printing, so maybe something run out of Memory but i dont know what wehre and why.

This is a memory-exhausted error. Assuming that this is a 1GB RAM Raspberry Pi 3B, this rarely happens. If you're running this on a 512MB RAM something else then this might be a cause.

If it were me, I'd run htop in a remote session and watch the amount of available RAM. I'm guessing that you're not running in Safe Mode and that something still has a memory leak which is draining away your RAM. The usual suspect would be the gstplayer video provider on the Pi for the webcam. Disconnect the webcam and see if the htop memory level doesn't climb.