Why does OctoPrint display files on the printer's SD so weirdly?

OctoPrint displays files from your printer's SD card in DOS 8.3 format because this is how it receives them from your printer's firmware:

>>> M20
<<< Begin file list
<<< MK3_PL~1.GCO 1292573
<<< MK3_PL~2.GCO 5812176
<<< MK3_PL~3.GCO 3190208
<<< MK3_PL~4.GCO 38102949
<<< MK3686~1.GCO 350775
<<< MK310C~1.GCO 1669802
<<< MK2E6F~1.GCO 9426676
<<< MK84CF~1.GCO 38935303
<<< MK57DB~1.GCO 20070962
<<< MK4CEF~1.GCO 23558921
<<< MK7400~1.GCO 456914
<<< MK9F5E~1.GCO 19026227
<<< MK6E26~1.GCO 29052944
<<< MKC88B~1.GCO 19661697
<<< LOWER_~1.GCO 3004206
<<< CORNER~1.GCO 12591171
<<< TOP_PA~1.GCO 1685493
<<< ONLY_H~1.GCO 53
<<< TRISPI~1.GCO 2255391
<<< 20MM-B~1.GCO 140760
<<< TOP_PA~2.GCO 1824381
<<< PRUSAT~1.GCO 2255391
<<< LITHOP~1.GCO 364087
<<< LITHO_~1.GCO 4578509
<<< LITHOP~2.GCO 5847248
<<< LITHOP~3.GCO 4829981
<<< LITHOP~4.GCO 9009070
<<< LAMP_B~1.GCO 3251150
<<< LAMP_B~2.GCO 3382527
<<< BASIC_~1.GCO 231093
<<< COVER_~1.GCO 342502
<<< OCTORE~1.GCO 3514373
<<< End file list

Some newer firmware variants out there support a command M33 to retrieve the long name stored along side the file on the SD for one single file. This is sadly not feasible to use by OctoPrint since it would have to send first the file list command M20 and then an M33 for each and every file stored on your printer's SD. That would pretty much lock up the serial connection to the printer for an undeterminable amount of time with n+1 request/response pairs.

So unless firmwares extend the M20 command with the long filename (which has been talked about but sadly never was implemented anywhere as far as I know) or provide an alternative that allows retrieving all long names from the SD, OctoPrint will have to keep displaying the 8.3 file names since blocking the serial communication just for file list retrieval is simply unacceptable default behaviour.

There are a few different ways to handle this.
First of all, it should be a user decision whether to "lock up" the serial connection or not. In my case I don't have that many files on my SD-card to really "lock up" the serial connection. So, there could be an opt-in selection to fetch long filenames. Secondly, an alternative cold be to click an icon on a specific filename to get the long name. Both of these could (should) be implemented in Octoprint.

A long time ago MS-DOS wrote files to FAT-formatted partitions and the size of the filenames were 8.3 (eight characters, an optional period and then three more characters).

The storage space required for long filenames means that the format of the partition needs to allow this.

The average Arduino + RAMPS board is running on limited RAM and probably for this reason the Marlin firmware, for example, chooses simplicity. It just uses FAT-based partition formats for the SD card and only writes/reads 8.3 filenames.

So—to the best of my knowledge—whenever you copy a file from your workstation over to the SD mounted in it the moment it lands on that FAT-based partition the long filename is lost.

Another reason is that Micro$oft forces vendors to pay royalties when they include code to write long filenames to a FAT32 partition. So I'm sure the industry is like meh, why bother?


What the industry should do is to standardize on a more Linux-like filesystem format and leave Microsoft behind. Then both Apple and Microsoft will have to chase the standard instead of the open-source industry trying to pay royalties to those two.

2 Likes

I'm eagerly awaiting your PR.

To add the necessary extension points to do this to OctoPrint of course, and then a plugin to make use of them which you'll meticulously maintain.

See, asking for something to be implemented is easy. Sometimes even the basic implementation is easy. But then reality hits and you get to maintain that for all eternity and consider all the different and difficult workflows your feature is now facing in the field.

2 Likes

"It just uses FAT-based partition formats for the SD card and only writes/reads 8.3 filenames."

That's not the case. Both my printers display long filenames from the SD card, so just like it says in the FAQ, the full names are there, just not fetched by Octoprint, for communication reasons.

Also, it is only writing of filenames that is subject to any royalty, in the extent that is really true/enforced, not reading. And those long filenames are indeed written to the SD-card by Cura.

I have to say I am surprised by the argument for not displaying the long filenames.

Have a look here: https://en.wikipedia.org/wiki/Filename#Length_restrictions

Your long file names belong to the M33 command foosel mentioned above.

foosel, that's an extremely arrogant response, I am sad to see it. If that is the attitude, then nothing would ever get done, because it would require maintenance.

I understand Octoprint has a "full time employee", paid by donations. Moving on from stone age MS-DOS filenames is not a big thing, rather the contrary.

This is a matter how Marlin wotks with the SD card that has to be FAT format. This is not an issue of OctoPrint. Every way to display long file names at the moment is just a workaround.

Said "full time employee" has to make tough decisions in order to keep the project running with the limited resources that are available to her - me. That includes weighing the usefulness and implementation and maintenance overhead of everything that's requested to be added to OctoPrint. The thing you are asking for here cannot be implemented in a sensible manner as things currently stand on the firmware side, as repeatedly pointed out now.

I'm open to accepting extension points in OctoPrint to allow others to invest the time of developing and maintaining a workaround using the above, but I'm not willing to implement something which I already know will only cause issues and more demands of me to make things work with even more workarounds that cannot be made to work reliably unless there are changes in firmware that were discussed ages ago but never attempted (as far as I know), taking time from way more pressing things that need to be fixed/implemented/solved. Demanding anything else of me is to demand of me to make decisions which I know will hurt OctoPrint.

And I'm frankly tired of constantly having to justify myself for refusing to build in workarounds for missing functionality on firmware side. Workarounds that I know will cause more issues than they'll solve. All I'm asking is that people who make these demands stop and think for a moment about what they are demanding and the implications of that. But sure... call me arrogant instead, that works too.

6 Likes

My attitude about open-source is the same as it is in real life. I wouldn't walk into a restaurant and ask them to create a special off-the-menu dish for me unless I also offered to pay for any hassle required. But I've done this. I've said "can you make a fettuccini alfredo with shrimp?" if it wasn't on the menu but I've seen that they have other dishes with the same set of ingredients. I'll then offer "just charge me whatever you think that would be worth" and they often will do so.

But I wouldn't ask anyone in the open-space world to add some big feature without also offering to pay for that work. Even on here, I've offered bounties for coding and tried to setup payment for same.

1 Like

Alright, so your firmware presumably uses a FAT32 format for the SD card and may not be the standard Marlin (perhaps you could identify it to continue this dialog). In theory, they have paid Microsoft royalties for doing so.

So next, it sounds like the M33 command in Marlin 1.1+ might support this on a per-filename basis.

The M20 command lists the contents. It might support the S2 argument which returns the listing in json and might also include the long filename support but perhaps that is just reprap.

A plugin could possibly be written which would accomplish this. I would value that work (for myself at least) at $1,000 for the initial iteration if a company were asking me to do this. I'm sure others might do it for less and some even for free.


Backing up from all this, though, just constraining the filenames back to 8.3 naming standards seems to be the easiest way of dealing with this.