Trying to Upload Files to Octoprint using Node Red and REST API


#1

What is the problem?
I am currently trying to upload gcode files to the octoprint local server using the octoprint api and node red. I have managed to POST a very small gcode file, 29 KB but when I try to POST another gcode file, 4 MB in size, I get a response on node red of:

"Error: write EPIPE"
or
"Error: socket hang up
both with
statusCode: "ECONNRESET"

the msg I am posting looks like this and I am POSTing it as a utf-8 string

msg.headers = {
    "Content-Type": "multipart/form-data; boundary=----WebKitFormBoundaryDeC2E3iWbTv1PwMC", 
    "X-Api-Key": "XXXXXXXXXX"
}
msg.payload = '------WebKitFormBoundaryDeC2E3iWbTv1PwMC\r\n'+
'Content-Disposition: form-data; name="select"\r\n'+
'\r\n'+
'true\r\n'+
'------WebKitFormBoundaryDeC2E3iWbTv1PwMC\r\n'+
'Content-Disposition: form-data; name="print"\r\n'+
'\r\n'+
'false\r\n'+
'------WebKitFormBoundaryDeC2E3iWbTv1PwMC\r\n'+
'Content-Disposition: form-data; name="file"; filename="'+filename+'"\r\n'+
'Content-Type: application/octet-stream

'GCODE'

'------WebKitFormBoundaryDeC2E3iWbTv1PwMC--';
  • uploading the file to octoprint using the regular octoprint GUI works fine
  • POSTing the file internally works (i.e. to another node in node red).
  • I am running node red and octopi on the same raspberry pi 3B+

What did you already try to solve it?

  • I checked the node red maximum HTTP request data volumes and timeouts and made sure they were not the issue - I am wondering if there is some kind of limit on octoprint api upload
  • Checked the octoprint.log: shows the file analysis for the small file but nothing for the larger one
  • Tried POSTing the file to a temporary location before POSTing it to the Octoprint server from that folder

Additional information about your setup (OctoPrint version, OctoPi version, printer, firmware, octoprint.log, serial.log or output on terminal tab, ...)

Octoprint Version: 1.3.10
Octopi Version: 0.16.0

Node Red Flow:

[{"id":"3f7e7ffa.4734","type":"ui_template","z":"d0969e5.ee64a6","group":"bc476fb4.91eae","name":"Upload HTML","order":13,"width":"8","height":"2","format":"<h1>Upload</h1>\n\n<form action=\"/temp\" method=\"POST\" enctype=\"multipart/form-data\">\n    <input type=\"file\" name=\"myFile\" />\n    <input type=\"submit\" value=\"Submit\">\n</form>","storeOutMessages":true,"fwdInMessages":true,"templateScope":"local","x":1380,"y":280,"wires":[[]]},{"id":"ae8c13a7.6bfe1","type":"function","z":"d0969e5.ee64a6","name":"Format the header and payload","func":"msg.headers = {\n    \"Content-Type\": \"multipart/form-data; boundary=----WebKitFormBoundaryDeC2E3iWbTv1PwMC\", \n    \"X-Api-Key\": \"XXXXXXXXXX\"\n}\n\n\nmsg.payload = '------WebKitFormBoundaryDeC2E3iWbTv1PwMC\\r\\n'+\n'Content-Disposition: form-data; name=\"select\"\\r\\n'+\n'\\r\\n'+\n'true\\r\\n'+\n'------WebKitFormBoundaryDeC2E3iWbTv1PwMC\\r\\n'+\n'Content-Disposition: form-data; name=\"print\"\\r\\n'+\n'\\r\\n'+\n'false\\r\\n'+\n'------WebKitFormBoundaryDeC2E3iWbTv1PwMC\\r\\n'+\n'Content-Disposition: form-data; name=\"file\"; filename=\"'+msg.req.files[0].originalname+'\"\\r\\n'+\n'Content-Type: application/octet-stream\\r\\n'+\n'\\r\\n'+\nmsg.req.files[0].buffer.toString()+'\\r\\n'+'\\r\\n'+\n'------WebKitFormBoundaryDeC2E3iWbTv1PwMC--';\n\n\nreturn msg;","outputs":1,"noerr":0,"x":1610,"y":360,"wires":[["e0a2faa3.61f9c8","af10b7df.746cb8"]]},{"id":"aec424f6.b420f8","type":"http response","z":"d0969e5.ee64a6","name":"","statusCode":"","headers":{},"x":1830,"y":300,"wires":[]},{"id":"c000a6a1.93de68","type":"http in","z":"d0969e5.ee64a6","name":"","url":"/temp","method":"post","upload":true,"swaggerDoc":"","x":1350,"y":360,"wires":[["ae8c13a7.6bfe1","729b8d86.ae6ca4"]]},{"id":"e0a2faa3.61f9c8","type":"debug","z":"d0969e5.ee64a6","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":1750,"y":440,"wires":[]},{"id":"729b8d86.ae6ca4","type":"change","z":"d0969e5.ee64a6","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"file uploaded","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1630,"y":300,"wires":[["aec424f6.b420f8"]]},{"id":"af10b7df.746cb8","type":"http request","z":"d0969e5.ee64a6","name":"","method":"POST","ret":"txt","paytoqs":false,"url":"http://racka/aa/api/files/local","tls":"","proxy":"","authType":"basic","x":2050,"y":360,"wires":[["91ed4e52.bdbd4"]]},{"id":"91ed4e52.bdbd4","type":"debug","z":"d0969e5.ee64a6","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":2230,"y":360,"wires":[]},{"id":"bc476fb4.91eae","type":"ui_group","z":"","name":"AA","tab":"86ac15ae.8d8e88","order":1,"disp":true,"width":"10","collapse":true},{"id":"86ac15ae.8d8e88","type":"ui_tab","z":"","name":"Overview","icon":"dashboard","disabled":false,"hidden":false}]

Any help would be much appreciated


#2

You might try adding something like:

Content-Length: 1325

...to the header to tell the server how much this is going to be. In the absence of the size of the file, the server probably wonders how long this is going to take.


#3

So, I tried putting Content-Length in the header which I got from the original POST request that uploads the file to the node red but I am getting the same issue I also tried with some arbitrary values. When I add the gcode to the payload though I guess I am (nearly) doubling the size of the request if I then POST that to the Octoprint server. I have tried removing the other parts of the original request so that all I am POSTing to octoprint are the payload and headers (_msgid is the same)


#4

Just searching for that term in the "post multipart ECONNRESET" space...

  • Somebody changed to an axios call for the win in one case. I have to assume that the promise aspect of axios could have helped.
  • Here's an issue from somewhere which also mentions ECONNRESET
  • If you're using the request module they mention trapping timeouts here

I dunno. If you're attempting to upload the file straight to the SD card on the printer, try just to the local system on the microSD card instead since this would be faster. In theory, the Content-Length should have made this happier if it's the correct size. If the file's name has any UNICODE characters in it, you probably have to decorate that filename=... part to adjust it to UTF-8, I'd imagine.

Your code looks right enough to me... except for maybe the apostrophe pairing here:

'------WebKitFormBoundaryDeC2E3iWbTv1PwMC\r\n'+
'Content-Disposition: form-data; name="file"; filename="'+filename+'"\r\n'+
'Content-Type: application/octet-stream

'GCODE'

'------WebKitFormBoundaryDeC2E3iWbTv1PwMC--';

The end of that Content-Type line needs to end with \r\n\r\n' +, etc. And your 'GCODE' likewise needs to be concatenated correctly.


#5

I fixed the code, I'll try those other solutions but I think I might shelve this one for now. Thanks for your help though.


#6

Feel free to describe how you fixed it, in case anyone else stumbles across this thread in the future.


#7

Ah, my last message was misleading, I made the code fixes you suggested but I still couldn't get it to work :sweat_smile:


#8

Fair enough. :slight_smile: