Communication Errors Cancelling Prints


#1

What is the problem?
Lately I've been having prints cancel because of, what I believe to be, communications errors. The first few failed prints had messages of "Printer requested line __ but no sufficient history is available, can't resend" where sent lines weren't received and portions of the received messages were missing. The past couple of failures were "Recv: start
Printer sent 'start' while printing. External reset? Aborting job since printer lost state."

I've noticed temperature variations when the communications errors start to appear before they rise to the level of cancelling the print. I'm thinking this is a voltage issue coming from the PS, but do not have an extra on hand to test. Replacing the power supply is my next step, if this seems reasonable and there isn't something else that should be tested first.

What did you already try to solve it?
I switched out the USB cable (both seem to be working, but both could conceivably have faults), disconnected my webcam, and put the Pi back on a dedicated power supply (had been on the ATX PS that powers the printer).

As a side note, this setup has been working fine since I last modified/moved it ~8 months ago.

Additional information about your setup (OctoPrint version, OctoPi version, printer, firmware, octoprint.log, serial.log or output on terminal tab, ...)

OctoPrint version: 1.3.9

Printer: modified Folger Tech Kossel running Marlin (would need to check the version)

Image of the temperature variations: Communication%20Error%202

Log files: serial (2).log (559.3 KB)
Have the octoprint.log, but can only include one attachment.

Thank you for any assistance!


#2

-First, try a print from your SD card. This isolates the comms to your boards. A good print rules out your boards. If it cuts out or you still have that temp fluctuation on that ONE thermistor you need to swap the thermistors plugs on your ramps AND the pin assignments in Marlin. If the noise follows to your bed thermistor then it's your board. Either it is shot or there is some powering fluctuation on that channel. If the noise stays with the hot-end thermistor you probably have something noisy that is running parallel to your hot-end thermistor cable.... and probably your USB cable...
-Isolate your buck converters from your boards and cabling. They produce a bunch of noise.
-Then check how all of your supplies are plugged in. Are they plugged into sockets that are on the same circuit; if so does that ckt also have an appliance that has a lot of frequent, intermittent draw? If you cannot easily mitigate the issue by simply moving your supplies to another ckt you need a line conditioner or full sine-wave UPS with integrated conditioning. $$$$$$$
-then spend a month looking at other modes of transport between the pi and your MCU like Ethernet, wifi, UART, basically grasping at straws because you can only guarantee your prints will finish if you use the SD card that takes 15 minutes to transfer your gcode file or you are unplugging and plugging in your SD card to file transfer (so bougie) but that defeats the entire idea of using the Pi in the first place so you spend some weeks researching and hoping that someone will harden the com protocol like XON/XOFF but you know deep in your :heartbeat: that it will not happen anytime soon so you research research research and come across klipper, you got NOTHING to lose so you make the plunge and you all fixed.... Oh wait sorry that was me.. ;
-this is a very common problem. Good luck.


#3

Thank you for the suggestions.

I don't have the ability to print from SD card directly on the printer, so I'm limited in that debugging. I also don't have buck converters, so at least that's one less thing to worry about. :slight_smile:

Originally, everything was running off of one ATX power supply, and when I swapped the Pi to its own, they were still on the same circuit. Nothing major is on that circuit and I can't think of any new devices or anything that changed over the time that this problem developed.

I looked over the wiring to run the USB cable as far away from motors/power wires as possible (though that didn't result in a big change) and double checked that nothing had pulled loose.

Before I start modifying the firmware (which means I need to find my most recent files), I figured I'd try another test print. It's currently running (40 minutes in as I type). I've seen some temp fluctuations, and some lines that have needed to be resent, but nothing that has caused the print to cancel yet. I currently have the Pi reconnected to the ATX power supply and updated OctoPrint to the latest RC in hopes that I'd get an under voltage notification if that was the issue.

Thanks again! I'll update when this print finishes or fails...


#4

Well, the print ultimately failed. I got the external reset message again.

Send: N839 G0 F3600 X-24.144 Y-42.529*70
Communication timeout while printing, trying to trigger response from printer. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
[...]
Recv: start
Printer sent 'start' while printing. External reset? Aborting job since printer lost state.

I was trying to keep an eye on the 12v line and regularly saw 11.8x and 11.9x volts, with occasional flashes of 11.6x. Unfortunately, I wasn't watching when it cancelled, but what I observed seems within tolerance.

The earlier resets were triggered when portions of communication were lost, and there were certainly a good number of missed lines in this print. However, the fact that OctoPrint received a "start" message makes me doubt wiring/interference unless those could cause Marlin to send that message. I assume a voltage drop would cause the Arduino to reboot and send the "start" message, but I'm unsure what other conditions could cause this.


#5

Make sure that your serial cable either has internal metallic shield or it has one or more ferrite cores.

If you're sure that's the case, if it were me I think I would try to add a stretchy ponytail to both ends of the serial cable (temporarily) so that they don't wiggle loose. If that's the case, then consider replacing or tweaking something so that the serial cable ends are tight. Note that the typical vibrations inside a printer can bump connections loose (especially the ground). A number of manufactured Raspberry Pi 3B computers had loose power connectors similarly so you have to be careful.


#6

I replaced the USB cable again (new one with much better internal shielding and a ferrite core on each end) and the power supply, and have been printing without any cancelled prints for almost 20 hours now. I did see some resend messages early on, but haven't noticed any in the past few prints. I haven't tried isolating the issue to the USB cable or the power supply yet... It's working and I don't really want to touch it. :slight_smile:. I'm planning a complete rewire over the holidays when I finish the enclosure and may do more testing then.

For now it seems to be working. Thank you everyone!


#7

Feel free to rewire, just enjoy a stint of good printing first to make sure that you've really quashed the problem.

I've seen really odd things like a microwave oven causing wi-fi problems, a cycling air conditioning unit lowering the line voltage, wi-fi interacting with a radiator and other really random stuff that you would otherwise ignore. Microwave ovens operate in the same frequency of wi-fi, btw.


#8

The upcoming rewire is completely separate from this issue, but I do want to get this squared away first.

I thought the issue was solved, but I was wrong. Prints with a larger footprint are routinely having issues still (bad character or two in the request to resend a line). Smaller base prints are working fine even if they are a longer print.

I connected the printer to my PC via USB and it's printing without any errors.

I have noticed as part of this that my Pi hasn't been starting up correctly the first time and tends to need at least one power cycle before connecting to WiFi. I haven't connected a display to it yet to see if that gives any insight on what is hanging. I haven't seen any undervoltage or overheating warnings.


#9

When you say "connected the printer to my PC via USB" I'm assuming that you cut the Raspberry Pi out of the picture here. If so, it feels like the Raspberry Pi or its power may be the weak link.

If you have a spare UPS, you might want to insert it into the picture here, powering everything involved from it. Sometimes it's the building power and has little to do with anything from one gcode file to the next. (It's like I mentioned with completely random things outside of your troubleshooting efforts still affecting you.) If the UPS makes the problem go away then it's your building power or more specifically, the wiring behind your outlets themselves which drop the voltage. If the building power is lower than it should be, this would make a 12V power adapter likewise be at undervoltage.


#10

Correct, I removed the Pi from the control role. It was still running, but only for the camera. The printer was connected via USB directly to my PC running Repetier-Server.

I do have an UPS that I could put in line, though I likely will not be able to test today.

My primary way of powering the Pi is off the 5v rail of my ATX power supply which is rated at 17A. I also have a " CanaKit 5V 2.5A Raspberry Pi 3 B+ Power Supply". While trying to diagnose the problem, I've used two different ATX PSs and the CanaKit PS to power the Raspberry Pi and all had the same results.

I should be able to borrow a Pi from a friend to test that component. Should I do a fresh OctoPi install for either the current or replacement Pi?

Thank you for all the help so far!


#11

I guess what I'm suggesting here is a holistic approach to power: begin with a multimeter at the wall plug and monitor the output voltage (it may surprise you). The UPS would be an attempt to rule out all the work that you've done on this side of the power issue.

I wouldn't do a fresh install yet.