I'd like to create an OctoPrint plugin to upload files to the printer's sdcard via a custom protocol. It appears that the hook, "octoprint.printer.sdcardupload", is the proper mechanism, but I've two issues:
It seems that I can get access to the serial port with printer.get_transport(), but not unfettered access to read and write. While the file is uploading, I need to "suspend" OctoPrint's normal serial processing that occurs in the "monitoring" and "sending" threads. It appears that setting flags, monitoring_active and send_queue_active, to False will terminate the threads, not temporarily suspend them. Is there a proper way to temporarily take control of the serial connection?
Also, what is the prefered way to indicate the file's upload progress in Octoprint?
All in all, my upload plugin should be a fairly straight-forward endeavor, and I think, will work well in OctoPrint. Hopefully, an OctoPrint expert can comment and get me over these two technical hurdles. I appreciate any help or insight.
You will probably have to disconnect the printer's connection and then reconnect once your process is completed. Something like the way the FirmwareUpdater does here.
Thank you! Your suggestion was spot on. I was concerned about changing the "printer state" (when closing the serial port), but OctoPrint seems to handle it more or less gracefully. Files upload smoothly.
I've not tackled the progress bar. It moves quickly to 100% and "spins" while the file transfers. Any pointers on how to properly access the progress bar?
Not sure if there is a callback method, but if your plugin sent a message (send_plugin_message from your python side) back to the web interface (onDataUpdaterPluginMessage from your javascript side) of it's progress you could use a bound filesViewModel dependency and use the same callback defined here.
It's really going to be dependent on if your process for transferring a file to the sd card is able to calculate a progress to send to the UI side. Assuming that is possible, I'd be happy to help if you were to share your code on GitHub or otherwise.
Mainly because I'm interested in faster SD card file uploads.
Thank you for your offer. Here's a little background and where I'm at with the plugin:
I ported Marlin firmware to the Monoprice MP Mini Delta 3d printer (mpmd_marlin_1.1.x). Along the way, I coded a simple upload mechanism as an alternative to the painfully slow (on the Mini Delta, anyway) upload procedure that Marlin employs. The firmware retains the original Marlin upload command (M28), and also supports a custom simple upload command (M990). My OctoPrint plugin takes advantage of this custom M990 command.
The first version of the plugin is complete and up on GitHub (ao_m990_upload_to_sdcard.py). The plugin works and is usable, but it does not update the progress bar on the OctoPrint client. The plugin includes an empty function, ao_set_progress(percent), that hopefully will handle the progress indicator in the future.
Though no speed demon, the plugin (and M990) is at least 10x faster than Marlin's M28 command, uploading at ~8Mbytes/min on my test machine. There are some other limitations (e.g. always writing to the root directory on the sd card), so it is far from a perfect solution.
That's too bad on the custom firmware front. Was hoping it was something native to Marlin. In either case, I took your example and packaged it into a plugin package with all the required stuff. I just ismply merged your file into the __init__.py so there may be some more work that needs to be done, but since I don't have the firmware can't test this out completely. You obviously know what you're doing from a python perspective, so hopefully you'll be able to figure out any quirks that need to be resolved. I've placed the plugin package here for your reference.
You can simply test to see if it works by using the URL below in plugin manager, but I doubt it will without modifications you may have to move some functions around, etc.
Again, thanks. I will use this to create a full-fledged OctoPrint plugin. In the meantime, I'll let the 1-file version out for testing and a bit of field trial. I can write the full-fledged plugin to "play nice" and use the M990 mechanism wherever it finds it. Who knows, maybe the M990 command will show up in mainstream Marlin.
You have been a big help. I appreciate it.
{ I think it will be very difficult to squeeze any more performance out of Marlin's file upload. The protocol pretty much works against speed and efficiency in favor of a myriad of checks to ensure that valid g-code is being uploaded. Not a bad idea, but overkill IMHO, as the underlying transports do a fair amount of checking all ready. }
I did look at that discussion some months ago. It is slightly similar in that transfers occur in fixed sized packets vs text lines. In the proposal you point to, there are underlying presumptions: 1) that data is binary, 2) that the protocol must be integrated with the existing protocol, and 3) that the communication link is inherently error prone -- this reasoning leads to a solution that (IMHO) is far more complicated than necessary. In essence, the Marlin discussion is trying to "solve" a larger (perceived) problem than the more constrained, "we need to upload gcode files faster". I went the other way: since the printer cannot do anything with files other than gcode files, I decided to focus on transfering gcode files to the sd card as fast as possible. The slightly different aims lead to the differing solutions.
Of course, we'll need see just how this M990 thing does in the field.
Cool, how hard is it getting your M990 support integrated into Marlin? I'm very comfortable with compiling firmware, etc. Have you considered contributing the changes back to marlin once additional testing is completed?
Marlin 2.x seems better organized to accept additions such as an M990 command. The technical challenges are similar to the plugin -- suspending and resuming Marlin's normal communication process, updating some kind of progress UI (user interface) on the printer, and detecting a "cancel" from the printer's UI. Technical issues aside, there may not be that much of a need. The discussion in Marlin's issue #382 seems to put this in the "meh, it would be nice" category (faster upload issue #382). I guess if a faster upload is popular on my port, there is a case to made to add it to Marlin in general. And an OctoPrint plugin may "stack the deck" in favor of its inclusion. We'll see.
I probably will fork Marlin 2.x at some point and make my MP Mini Delta changes/ improvements available through pull requests. I still need to understand better Marlin 2.x's organization and truthfully I have some ideological differences with some of the coding guidelines (practices that I believe make the code much more difficult to understand). On the technical side, some of the low-level board specific things diverge from the "Marlin way" as well. So, there are some things I need to work out before I "dive" into a Marlin 2.x port.
I point out, too, that before I started in on my Marlin port (mpmd_marlin...), I spent about three months trying to understand Marlin and the Marlin4MPMD projects, culminating in just trying to successfully build the projects via a Makefile. { editorial: I am not a fan of (development) systems that try to absolve the developer of the responsibility of understanding how and why the pieces fit together. } Turns out, that having a command line build (via a Makefile) simplified introducing automated builds and deployment with Travis-CI. Of the successful elements of the project, it is an important aspect to being able to share my work. I'm adhering to a similar process before attempting a Marlin 2.x fork.
