OctoPi losing network connection mid-print

Im seeing massive errors in the system log that refer to Broadcom Mac :

Feb 18 16:22:51 octopi kernel: [24922.329744] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.329794] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.329839] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.329882] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.329923] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.329964] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330005] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330047] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330088] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330129] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330173] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330214] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.330258] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.331106] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.331809] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
Feb 18 16:22:51 octopi kernel: [24922.331976] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -5
sudo modprobe -r brcmfmac

Will remove the wlan0 adapter and stop the errors.... however

sudo modprobe brcmfmac

will not add it back.

It's a bit of a sledgehammer approach, and if the wifi chip/module has frozen or faulted up at a low enough level it's just not enough. This works about 90% of the time for my rpi4, but sometimes it needs an actual power cycle.

This is a software issue. This device has been on my network for literally months before I flashed it to OctoPi.

Firmware looks the same:

Working:

pi@raspberrypi:~ $ ls /lib/firmware/brcm
BCM43430A1.hcd            brcmfmac43242a.bin        brcmfmac43436-sdio.clm_blob  brcmfmac4354-sdio.bin
BCM43430B0.hcd            brcmfmac4329-sdio.bin     brcmfmac43436-sdio.txt       brcmfmac43569.bin
BCM4345C0.hcd             brcmfmac4330-sdio.bin     brcmfmac43436s-sdio.bin      brcmfmac4356-pcie.bin
BCM4345C5.hcd             brcmfmac43340-sdio.bin    brcmfmac43436s-sdio.txt      brcmfmac4356-sdio.bin
bcm43xx-0.fw              brcmfmac4334-sdio.bin     brcmfmac43455-sdio.bin       brcmfmac43570-pcie.bin
bcm43xx_hdr-0.fw          brcmfmac4335-sdio.bin     brcmfmac43455-sdio.clm_blob  brcmfmac4358-pcie.bin
brcmfmac43143.bin         brcmfmac43362-sdio.bin    brcmfmac43455-sdio.txt       brcmfmac43602-pcie.ap.bin
brcmfmac43143-sdio.bin    brcmfmac4339-sdio.bin     brcmfmac43456-sdio.bin       brcmfmac43602-pcie.bin
brcmfmac43236b.bin        brcmfmac43430a0-sdio.bin  brcmfmac43456-sdio.clm_blob  brcmfmac4366b-pcie.bin
brcmfmac43241b0-sdio.bin  brcmfmac43430-sdio.bin    brcmfmac43456-sdio.txt       brcmfmac4371-pcie.bin
brcmfmac43241b4-sdio.bin  brcmfmac43430-sdio.txt    brcmfmac4350c2-pcie.bin      brcmfmac4373.bin
brcmfmac43241b5-sdio.bin  brcmfmac43436-sdio.bin    brcmfmac4350-pcie.bin        brcmfmac4373-sdio.bin
pi@raspberrypi:~ $

Octopi:

root@octopi:/home/pi# ls /lib/firmware/brcm
BCM43430A1.hcd		  brcmfmac43242a.bin	    brcmfmac43436-sdio.clm_blob  brcmfmac4354-sdio.bin
BCM43430B0.hcd		  brcmfmac4329-sdio.bin     brcmfmac43436-sdio.txt	 brcmfmac43569.bin
BCM4345C0.hcd		  brcmfmac4330-sdio.bin     brcmfmac43436s-sdio.bin	 brcmfmac4356-pcie.bin
BCM4345C5.hcd		  brcmfmac43340-sdio.bin    brcmfmac43436s-sdio.txt	 brcmfmac4356-sdio.bin
bcm43xx-0.fw		  brcmfmac4334-sdio.bin     brcmfmac43455-sdio.bin	 brcmfmac43570-pcie.bin
bcm43xx_hdr-0.fw	  brcmfmac4335-sdio.bin     brcmfmac43455-sdio.clm_blob  brcmfmac4358-pcie.bin
brcmfmac43143.bin	  brcmfmac43362-sdio.bin    brcmfmac43455-sdio.txt	 brcmfmac43602-pcie.ap.bin
brcmfmac43143-sdio.bin	  brcmfmac4339-sdio.bin     brcmfmac43456-sdio.bin	 brcmfmac43602-pcie.bin
brcmfmac43236b.bin	  brcmfmac43430a0-sdio.bin  brcmfmac43456-sdio.clm_blob  brcmfmac4366b-pcie.bin
brcmfmac43241b0-sdio.bin  brcmfmac43430-sdio.bin    brcmfmac43456-sdio.txt	 brcmfmac4371-pcie.bin
brcmfmac43241b4-sdio.bin  brcmfmac43430-sdio.txt    brcmfmac4350c2-pcie.bin	 brcmfmac4373.bin
brcmfmac43241b5-sdio.bin  brcmfmac43436-sdio.bin    brcmfmac4350-pcie.bin	 brcmfmac4373-sdio.bin

That would usually be expected - OctoPi is Raspberry Pi OS Lite with OctoPrint pre-installed on it. All the firmware & stuff should be exactly the same. The OctoPi image was originally built early 2021 (now with an updated kernel/bootloader), so it will be a bit different than the current RPi OS images which may have seen updates.

You could try an OctoPi nightly build to test if RPi foundation have fixed the issues recently (the firmware update that was supposed to fix it still has some issues) - the builds are in the RPi imager selection or linked from OctoPrint.org - Download & Setup OctoPrint.

What I am saying is that my existing RPi4s witht he same firmware are rock solid for months.

And the point above is that OctoPi is the same operating system as well. The only difference between OctoPi and Raspberry Pi OS is that OctoPi has OctoPrint and associated apps preconfigured.

If you are still absolutely dead set on this being an OctoPi issue then you should install your OS of choice on the pi, and follow the linux install guide. I suspect you'll still run into these wifi issues intermittently, like several other users of rpi4's, myself included.

Ah awesome, thanks for the clarification that it does happen after a "vanilla" install.

Is this limited to Wifi? I assume so and its tied to the Broadcom failure log entries (brcmfmac: brcmf_sdio_htclk:)

I may see if I can write a script that tail-f's the log and waits for a substantial number of these errors and then saves the last few pages to see if theres something in there that hints as to whats going on

(My 250GB SSD that is running my rig had several dozen Gigs of system log by the time I caught it.)

As far as we know, yes. No one has reported similar issues when using cable connections, only seems to be related to the wifi driver

Awesome, Im going to try catching the system/broadcom logs.
I notice my CPU was throttled... did we get any feedback on folks getting in via ethernet before rebooting and seeing this?

If you have seen this issue, please fill out this poll to help:

You'll probably find more info here: RPi4: WiFi client crashes often (brcmf_fw_crashed: Firmware has halted or crashed) Ā· Issue #3849 Ā· raspberrypi/linux Ā· GitHub

I have this same exact issue. Similar to other users here I am using a Unifi setup, and my controller reports that the device is active and connected... but I can't connect to it unless I force a reconnect from the Unifi controller. This is the only device where I've ever seen a problem like this.

I am also on unifi and using "reconnect" does indeed reconnect the pi. For now I guess I will just use the script provided by @egar

Issue started for me this week after I made a change to my Amplifi HD WiFi Settings. I had disabled 802.11v. Reverting back and will monitor.

If you guys continue to experience this a lot you may want to try the PowerFailure plugin that was recently released, it works for disconnects as well.

1 Like

Experienced this Toay for the first time 10% into a Print.

It may also have happened yesterday on a couple of occasions too, as I had a couple of retarts while experimenting with things & though the Pi wasn't running.

On all occasions I could not ping it & it wasn't showing up in my Omada Wifi management.

I know it's a Pi issue, not an Octo Print issue, but will still comment on what I changed yesterday that caused the first problem; which at the time I didn't investigate to well.
Just power cycle'd it & then assumed it was fine.

The thing I changed yesterday though, was to enable I2C & installed the Micro Panel Plugin.

Disconnected the Display I bit later as mine was an SSD1106 & not compatible, but left the config unchanged & ordered a compatible screen.

Can't check a lot right now as a 4 hour print is in progress & I have got control back by plugging in an Ethernet cable & logging in thru it's IP.

Will probably end up leaving it wired; not point in going back; just a matter of routing a cable.

Cheers.

If I remember correctly, Ethernet connection will also stop working randomly not just WiFi.

People are saying plugging in Ethernet ā€œwakes upā€ or ā€œrestart driverā€ which is an alternative way to get WiFi to revive without having to reboot the Pi.

Latest AmpliFi Alien, I am not sure what fixes the issue. Pi or AmpliFi. It seems mostly stable. Maybe only restart once a year or less.

If Pi hasnā€™t made new changes to Networking driver, Iā€™m still concerned cuz we all donā€™t know what exactly is the problem.

I think problem is there are several different issues with similar symptoms, for me the WiFi stack crashes, so Ethernet allows it to connect, Ethernet is not flakey as pis on Ethernet have not been rebooted for years.

It also means the ā€˜wakeā€™ up methods donā€™t work for me as the stack is dead, in theory by removing and reinstalling the dynamic modules for WiFi it could be brought back to life but that is just to much like hard work, easier to reboot.

Lived with this issue fir a couple of years. For me the best solution is the reset button option

Pretty sure that's what the Network Health plugin does.

Unfortunately not, that just effectively does a stop/start network and fails because the kernel network wifi stack has died in a horrible way. You have to rmmod and then insmod them back in again (according to various forums), but I have never managed to get them to do it in an order that satisfies dependency. It's a known issue. Rebooting is easy.

I do have the network Health plugin for a year or so installed, before that I had the corn version but both fail to restart the network, you can watch it fail when you plug in via ethernet, hence my comment that many similar symptoms but different underlying issues.

I unfortunately don't have any good news to add but I too have been dealing with this issue on and off for a few years but recently it's been getting worse. What happens to me is a combination of things. I leave my server on 24/7 and only turn off the printer. Server is on a pi4 with active cooling. Nothing else running on it. I'll list the two main issues I have been having.

  1. After a seemingly random amount of time the server will loose wifi connection and not re-connect until reboot.
  2. About 50% of the time WIFI will stop working a moment or two after I start a print. The print doesn't stop, in fact the print has always continued and completed without issue. But again I will only re-gain wifi if I reboot the pi manually.

I have ruled out the router (tried multiple routers through the home). Signal strength (for a test moved the printer next to the router). Static/Dynamic IP (this made no difference with the above issues). It's not a bad pi (tried more than one). It's not corruption of the build (formatted and re-installed octoprint multiple times). It's probably not the printer (same issue, I have two printers).

I just want to note that similar issues have been noted in this thread and many other on several forums for a few years. So I believe the issue has yet to be found and may not be wide spread because in some cases it may not be a bug at all. The one thing almost all have in common is a hardline fixes the issue.

But given the lack of fix and similarity of issues among a smaller group of people I am theorizing the issue may have more to do with specific conditions. For example high static, electrical noise from printer/power supply ect.. I plan to do some tests including further isolating the pi and various changes to reduce electrical noise and see what happens. If i find anything significant I'll post my findings.