Octoprint shows ERROR ON UPLOAD with FSTAB and mounted external storage

I'm working on a Pi 3+ system. Boot is done by SD card like standard. But data are stored on a USB drive, mounted on boot automatically by FSTAB.
Works well.
But if I upload files using Octoprint always an error message ERROR ON UPLOAD pops up and the automatically update of the file list doesn't work.
But the file is stored proper to USB drive and if I do a rescan of Octoprint upload directories by pushing the 'reread button' the new file is shown in the list and I can print it.

I guess there is a problem with the 'directory change watcher' inside Octoprint.
Does anybody know a solution for this, maybe by a small code hack or a configuration setting?

Ok, it's more cosmetics but would be nice, if this error will happen no more.

Could you share your octoprint.log please? Also, how are you uploading? Through the web interface/API or through the watched folder?

Upload is done using Octoprint web interface.
The log gives very little information - 1 line only:
2020-11-06 19:32:28,467 - tornado.access - ERROR - 500 POST /api/files/local (192.168.+++.+++) 550.53ms
The + I wrote :wink: Original IP is right.

Original web interface error message:
To make it better readable, I replaced the HTTP TAG markers by - (minus)
-title-500 Internal Server Error-/title-
-h1-Internal Server Error-/h1-
-p-The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.-/p-

Can you upload the entire log file? In order for the 500 error to happen, there has to be an internal error, which would be logged.

That's what the </> button is for - it is called 'Preformatted Text' so that it is not rendered, so you can use that instead of the extra work.

of course I could.
But believe me, there is no more information then

2020-11-06 19:32:28,467 - tornado.access - ERROR - 500 POST /api/files/local (192.168.+++.+++) 550.53ms

around this event.

If you tell me, what information from Octoprint init or whatever you like to have, please tell me.
And if there is a possibility to increase loglevel, I can give that a try.

I surely can assist you. You are talking to a C/C++ system programmer with 30 years of experience at work.

Maybe this log entry in another context could be helpful:
It was written to the log on starting a print job:

2020-11-06 09:23:18,065 - octoprint.filemanager.storage - ERROR - Error while writing .metadata.json to /mnt/usb1/octoprint/uploads/KabelSpooler
Traceback (most recent call last):
  File "/home/pi/OctoPrint/venv/local/lib/python2.7/site-packages/octoprint/filemanager/storage.py", line 1583, in _save_metadata
    f.write(to_bytes(json.dumps(metadata, indent=4, separators=(",", ": "))))
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/pi/OctoPrint/venv/local/lib/python2.7/site-packages/octoprint/util/__init__.py", line 1105, in atomic_write
    shutil.move(temp_config.name, filename)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 100, in copystat
    os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/usb1/octoprint/uploads/KabelSpooler/.metadata.json'

Sounds like a permission issue, accessing the external drive. Since it seems you installed OctoPrint manually (I had to guess this due to the path to the file in the error) check what the user permissions are.

Have you never needed a log file? We don't spend all the time making things make logs for nothing, it is most helpful when people just upload the full file, since it paints a full picture. At the moment it is like trying to guess the rest of the painting from a single brush stroke.

The log file contains important information about what events have happened so far in the life of the OctoPrint install. And since you didn't provide much information about your system, it would have shown that too.

At this point, it is just wasting our time back and forth with 'can you upload the file'. I apologise if you read the log and decide that it is not important, but far too many people's problems become entirely clear from logs, for example the other error you posted - if we got the log when you posted this topic originally, you may have a solution or at least some idea of what the problem may be.

2 Likes

Charlie keep calm.
Digging in log files is daily work to me. I just want to prevent unnecessary, time consuming work to you.
Permission issue - maybe. But on system level the Octoprint area on the USB drive is set to acess mask 0777 == all rights for everyone.
System is manual installed. Octoprint runs ahead Stretch

2020-11-04 08:26:30,998 - octoprint.environment - INFO - Detected environment is Python 2.7.13 under 
Linux (linux2). Details:
|  hardware:
|      cores: 4
|      freq: 1400.0
|      ram: 917016576
|  os:
|      id: linux
|      platform: linux2
|  plugins:
|      pi_support:
|          model: Raspberry Pi 3 Model B Plus Rev 1.3
|  python:
|      pip: '20.2'
|      version: 2.7.13
|      virtualenv: /home/pi/OctoPrint/venv

so the full path here and all sub folders have that same permissions mask? I agree that the error does appear to be permissions related.

Edit: maybe adding explicitly defined permissions of the user that is configured to run octoprint would help for some stupid reason.

yes so it works. I controlled it. Accessmasks are set propper.

good idea. I'll give it a chance

what's wrong is found. But actually I've no idea how to solve it without changing some basic system configuration.

Error is thrown by python2.7/shutil.py by function copystat(src, dst)
source file #100
if hasattr(os, 'chmod'):
os.chmod(dst, mode)

hasattr(os, 'chmod') doesn't care about destination drive. It checks operation system only.
Operation system is Linux, so hasattr() will return TRUE, because changeMode is supported.
But destination drive format is vfat, not ext* or other format, what supports Linux access modes directly on file level.
Because of that, chmode() fails, what is reported by the error message "upload failed".

Any other ideas then changing file system of USB drive, what would be not so good?

solved
... maybe a little bit illegal ...

I just patched the python shutil.py by wrapping the call of os.chmod() by a try-catch block.
Now it is

       try:
            os.chmod(dst, mode)
        except:
            pass

success - it works.

Unfortunately it would be overwritten by a possible python update. But I don't think, that Python 2.7 will be updated.

There's a much more straightforward way, just set the 'quiet' option in your fstab entry for the vfat drive:

/dev/sdb1 /usbdrive vfat noatime,flush,nofail,user,umask=000,quiet 0 2

This suppresses chmod / chown errors on the OS level.