I decided to post this to General as it is not a support issue as such, more like looking out if it's just me.
the symptoms:
After a print job finished a browser message pops up that connection to the server was lost. Reloading the page gives: host unreachable (ERR_ADDRESS_UNREACHABLE).
Ping fails. Waiting a while helps nothing. The raspi has power, the fan runs, I see the red light on and nothing of the green led.
I have to powercycle the raspi, it boots up and I search the logs with no hint as to what has happened.
Those issues occurr sporadically with many or dozens of prints in between that are not followed by a freeze.
the system:
raspberry 3B Rev 1.2
octopi 0.18.0
octoPrint 1.6.1
75% free space on sd-card
voltage, temperature etc monitored
cabled eth-connection
The raspi has been running constantly for years.
the logs:
The print host ist loosely integrated into my openHab2 home automatisation and openHab logs some key events of octoPrint, listening to the mqtt-broker running on the printhost.
This actually gives the best view of the timing of the issue.
openhab.log (oh server):
2021-08-16 14:13:38.133 [INFO ] [lipse.smarthome.model.script.mqParse] - tokoActive ist ON mit tokoPhase; bedCooling und tokoStatus: Progressing
2021-08-16 14:13:54.595 [INFO ] [lipse.smarthome.model.script.mqParse] - tokoActive ist OFF mit tokoPhase; bedCooling und tokoStatus: Finishing
2021-08-16 14:13:56.820 [INFO ] [lipse.smarthome.model.script.mqParse] - tokoActive ist OFF mit tokoPhase; bedCooling und tokoStatus: Operational
2021-08-16 14:18:56.107 [INFO ] [lipse.smarthome.model.script.mqParse] - tokoActive ist OFF mit tokoPhase; Ending und tokoStatus: Operational
2021-08-16 14:20:28.381 [INFO ] [.reconnect.PeriodicReconnectStrategy] - Try to restore connection to '192.168.99.48'. Next attempt in 60000ms
2021-08-16 14:20:28.569 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '192.168.99.48' with clientid oH2tieke
So, 14:18:56 the print has ended and octoPrint is ready to start the next and 94 seconds later the timeout for connection to the mqtt-broker triggers and openHab tries to restore the connection. As the timeout is 60 sec. that leaves a window of 34 seconds between the last successful message and the freeze.
event time is roughly 14:19
after reboot I ssh into the print host ("kiwi") and check /var/log/
syslog
Aug 16 14:02:01 kiwi systemd[1]: systemd-hostnamed.service: Succeeded.
Aug 16 14:58:21 kiwi systemd-timesyncd[304]: Synchronized to time server for the first time [fdec:db8::1]:123 (blesse.intern).
nothing.
kern.log, messages : no entry at all for the time frame.
octoprint·log
2021-08-16 14:16:31,352 - octoprint.server.heartbeat - INFO - Server heartbeat <3
2021-08-16 14:17:57,598 - octoprint.server.util.sockjs - INFO - Client connection closed: ::ffff:192.168.99.18
2021-08-16 14:01:32,167 - octoprint.startup - INFO - ******************************************************************************
2021-08-16 14:01:32,171 - octoprint.startup - INFO - Starting OctoPrint 1.6.1
2021-08-16 14:01:32,175 - octoprint.startup - INFO - ******************************************************************************
Again, nothing. (Don't let the timestamp fool you, it is off by ~1hr before the timesync is through)
network-pinger.log:
I have a tiny daemon running that pings the gateway every 6 minutes and logs the success, OK if it can get a ping back, FAILURE if it can not.
16.08.21 14:12 - OK
16.08.21 14:17 - OK
16.08.21 15:02 - OK
If what I experience was a network problem it should have logged at least 3 FAILURE lines. But there is nothing.
I set up that daemon specifically to test whether
a) the raspi was alife but unable to communicate with outside or
b) dropped dead. The latter appears to have happened.
the history:
The first time this ever happened was shortly after I followed the multicam-guide to run an usb-cam secondary to the raspi-cam.
I suspected this was somehow causing the freezes and rolled it back. Now a pi2 serves raspi and usb cam in parallel and has no problem doing that whatsoever.
The freezes occurred like every 3-4 days for a while. Then less often (not sure what changed).
I noticed over time that often a print had just finished before the issue occurred. (But I only now discovered that my openHab.log documents those things, too, so by now the connection between those is only anecdotal).
Then I set up the network-pinger, inspired by a very similar script I found in a thread here about wifi problems mid-print (the link I cannot find again but it is a different issue anyways).
That was about a month ago. Today it happened again.
What could possibly make a raspi freeze/die like that, without even the slightest trace in the ¸logs?
A memory leak? Should I try to reboot the box every other day?
A hardware issue like, say, a faulty memory chip?
It is not a showstopper thanks to the timing - after a print finished. It still bugs me.