You are expecting that there's no overhead at all involved here for actually getting the data to the printer, which it is however. The only way to get data on the SD is via the same kinda protocol that's used to print on your printer. So line numbering every single line, checksumming it, sending it over, waiting for the firmware to parse that line, check line number and checksum, stripping that, writing it to the SD, acknowleding this via ok, then sending the next line, ...
Yeah, it works how I thought it did thanks. Sure, not before using Octoprint, but pretty soon after.
Definitely what was happening was Octoprint was doing something, maybe as simple as asking for a list of every file on the printer which the printer was having issues with. Without Octoprint attached, no issue, so regardless of where the problem actually was, it was only coming to life with Octoprint attached.
Summary of my issue: Transfer of files to SD card was hanging, crashing, killing the printer / Octoprint. Files on SD card not showing in the list of files.
Resolution: Cleared all files off the SD card, aside from a few known good .gcode files.
I am fairly certain there was a file on the SD card that was causing issues when Octoprint requested a list of files on the SD and killing the printer. Looking in the file list on boot, it was not showing anything on the SD. Once I cleared most of the files, it started working OK and would list the files on boot and I could transfer files to the SD.
Note: Transfer of files to the SD card is extreemely slow via Octoprint, a roughly 13MB file would take about 3 hours. This is expected. As long as you can see the file transfering and list of files stored on your SD card, then the transfer and connection to the printer is still working.
Thanks everyone for the help, info and suggestions. Much appreciated.
Nah, stock clocks. Pretty sure the HW is fine too, as I stuck RetroPie on a spare SD card and that runs like a champ.
It's just a file (or files) on the SD card in the printer that was causing it.
When I start Octoprint now, I see a list of files on the Pi and on the printer's SD. Before, I was only seeing files on the Pi... nothing from the printer. This would have been trying to get the file info on boot and had already started what would go on to kill my printer and crash OctoPrint shortly thereafter.
(Boot up, Get a list of files on printers SD... invalid character / contents or something... , maybe the printer or maybe OctoPrint could not interpret something to do with that file, tried and tried and some bug caused a crash / hang / kill after a few minutes of trying). Something like that.
The fact that I was trying to copy files to the SD card was largely irrelevant, but did actually accellerate that issue... though it would crash anyway after a mere couple of minutes.
Anyway, the copy to SD is now working successfully and I have stability (which is ultimately what I was looking for), though now I can see just how slow that copy is - I sadly won't be using it.
It certainly wouldn't be the first printer we've seen freaking out due to some sort of corruption on its own SD, there are a couple of topics around these forums of printers starting to beep, reboot or do god knows what on file list.
If it's not a rare occurence, it might be worth putting a one liner on the " Setting up OctoPi" page, because the issue is not obvious (it doesn't tell you straight out that the printer is being killed because it failed to read a file etc.).
A simple one liner of "We reccomend clearing all files from your printers SD card before first use" would do it.
It's rare. It's just not "I've never heard about this before". And it seems to be somewhat printer dependent, all printers this has happened with so far that I'm aware of are Creality ones.
Might well be because the Creality comes with a bunch of files on the SD card when you get it. It's not an issue if you are using the SD card in the printer as it seems to only look at files when you tell them to open and can browse / list everything OK.
I have personally given up with using the SD card and just let OctoPrint do the whole job.
Apparently, if you initiate the SD print from Octo it will keep going even if Octoprint reboots etc. However, I found that working with the file on the SD card was just as slow as copying and so basically unusable. I might have been doing it wrong (by hitting load before printing it) but anyway, I didn't use that method.
You wont be able to use the layer change timelapse on the SD however, that only works when it is on Octoprint I believe (certainly what the warnings say).
Slightly correct, but horribly misleading with conditions.
If you start a print on the SD card, yes you can turn off your octoprint machine and the printer will continue printing (maybe).
If you reboot your octoprint machine, any attempt to reconnect to the printer can reset your printer, this is due to how many printer controller boards are made, based on arduino technology.
If you plan to turn off your octoprint machine, I would highly suggest unplugging the usb cable, then start the sd print from the printer's own control interface.
Conditions and restrictions apply, and there are always exceptions to the rule. Some boards have a jumper that disables reset on serial activity, some boards have it permanently disabled (these boards may also be the same boards that require external hardware to update the firmware).
Are there any plugin hooks available for the upload process? I'm thinking of some of the newer controllers (like the Re-ARM) that allow direct mounting of the SD card via USB. You do need to send an M22 to "unmount" the card from Marlin, mount the card, do the upload, unmount the card and send an M21 to let Marlin know it's back when the upload is done, but if there were plugin hooks available for this it could allow for much faster upload to SD card on this sort of controller.
In my humble opinion, the entire Marlin SD-card-formatted-as-FAT is just a bad idea. It's all 8.3 filenaming which is a long-dated concept. But it seems to be constrained by the low end of printer boards to available memory. Honestly, I don't trust the SD filesystem storage-related code in Marlin.
The AVR processor at the core of most Marlin installations has less
memory than the MS-DOS machine you probably had circa 1990. That's why
it uses the DOS "FAT" filesystem, which has 8.3 filenames and is very
easy to decode.
The longer filenames are stored in a separate name translation file.
That separate file can potentially be quite large, so practically
speaking, Marlin can't read the whole thing into memory for easy
reference. So in order to decode a long filename, it has to start over
at the beginning of the translation file and look at all of the entries
in there until it finds the one it needs to display. Then it would have
to repeat all of that work for each subsequent filename. On large
directories, it would be palpably slow.
A number of hacks can speed it up, but they make the code even more
complicated and delicate, and no hack can actually be efficient for large
directories. It's very tempting to go with one of these hacks, but then
the first time someone has a directory with 100 files in it, the hack
falls all over itself, that's why I call it a hack.
This is a recurrent philosophical question in Marlin development,
especially because there are more powerful ARM-based boards beginning to
proliferate.
But on the AVR boards, Marlin really has to act like an embedded system.
Almost all of the stuff for user convenience (like long filenames, sorted
filename lists, total extrusion length computations, etc) should really
be done on the host PC. The robot translates a G-code file into physical
motions and that is all. It doesn't have capacity to push around
thousands of bytes at a time like every other computer in your life can.
The AVR is much less than your cellphone, or even a modern car stereo.
