Is your feature request related to a problem? Please describe.
I'm working on changes intended to give the File Manager a new "single folder" view. So it will have a current-working-directory, up and back buttons, a header with the current path in a drop-down, and all that fun stuff. I'm taking this as an opportunity to get familiar with Python and the OctoPrint codebase while also trying to do "real development" on a Pi 400.
Before I could dive into the File Manager I had to expand the scope of my plan a little bit. Before I go too far into left field and start sending a parade of pull requests, I thought I had better first write up a "brainstorming" ticket and show the direction I am going so far. A link to the compare page is below, but here is the summary of changes I've made up to now:
Virtual Printer:
Simulate M154 S using RepeatedTimer.
Add AUTOREPORT_POS capability to the M115 report.
Simulate M420 T V (just T0 output so far), just returning a random mesh.
Modify _mappedSdList to return a deep list of the virtualSd folder contents.
Terminal:
Add a new Terminal Filter for M154 S reports and M114 replies.
Add AUTOREPORT_POS to the M115 feature queries, signaling M154 S.
I would like a File Browser with a single current directory, or any other view style which is easier to read when there are many files. The work in progress will still receive the full DOS 8.3 listing, but will gather all the unique paths and arrange them hierarchically as 'folders' in the view. Clicking a folder will change the view to show its contents. Clicking "up" will go up a level. History buttons may also be included. The display of the current path needs to be compact, so a simple drop-down is being considered.
Describe alternatives you've considered
I may also work on this as a separate plugin to take advantage of the wider area. Then it would be possible to do a columns view or a hierarchical folder view on left in the style of File Explorer.
Additional context
The changes related to M154 S will support Marlin 2.0.8.1 and are due anyway, so I expect I can separate that part out into its own PR. The changes for M420 V can also be made into a separate PR. They're included just to make sure that I'm following the right course.
My main concern with the deeper file listing is how it might affect any other things that call it. The file listing style should be digestible for anything that can parse M20 lines, so the only question is whether they are bothered by the full path being returned in the dictionary.
Back to the main issue, finally... Is it going to be difficult to add a new view to the File Manager, or is it actually pretty easy to add a new view like I have in mind? As a Python newbie, is there anything subtle I should be looking out for as I delve into that area of the code?
So, you are right @thinkyhead, this is actually a couple of separate issues.
Virtual Printer support for M154, AUTOREPORT_POS, M420 and SD deep listing
That sounds fine and would certainly be accepted.
Terminal filter for M154 and M114
M114 definitely sounds like it would be encountered often enough this makes sense to include by default, especially with enabled autoreporting. Don't see why you'd need to add a filter for M154 as that - as far as I understand it - only enables/disables position autoreporting and so should rarely be even seen in the terminal.
Commlayer tracking of position auto reporting capability & issuing of M154
On my own todo list, so definitely welcome. Cancel/pausing handling might need special attention here as that is currently waiting for an M114 issued after an M400. Would need some care to make sure cancel and pause position aren't recorded before movement has stopped.
Maybe you can provide some insight here as to how the autoreport position behaves... is it the exact current position, e.g. right in the middle of a move, or the current position of the last fully executed command? How does it interact with M400?
Filebrowser changes
This is where you've admittedly lost me so far. I don't fully understand what exactly you want to modify where. The file browser per se is the GUI element in the left sidebar. Your description however makes it sound like you actually want to change SD card file list parsing from M20? Do you propose to visually represent local and SD files in a hierarchy like this?
I read it as terminal filter for position reports, so exactly like the temperature filters but just for positioning.
The FileManager plugin adds some additional functionality, but a lot of that is now implemented in core anyway. The way I'm reading this, is to create more of a desktop OS style filemanager rather than a single list of files. I'm not entirely clear on what this would look like either, but my first thoughts are that it would need to move out of the sidebar to really become useful. Maybe 'quick access' in the sidebar, more detailed tab? Not sure. The sidebar is already pretty cramped as it is for the amount of info there.
That's how I understood it too, but I would expect position reports autoreported after enabling that through M154 to look the same as those queried by M114.
Possibly, but then the M20 comment and the "deep listing" change on the virtual printer threw me off. That's why I'd like to get clarification here first.
Thanks for moving my work-in-progress discussion to a better location. I have been separating out the M114 Filter and the Virtual Printer enhancements into separate branches and I'll submit those as individual PRs soon. We can discuss the substance of those after I submit them.
As for my interest in the File Manager, I want to modify it so that it can show folders in the view, and will allow you to double-click a folder to enter the directory, and do all the usual things. I thought I might need to modify the FM code itself, but I just saw the animation accompanying the…
…and based on that it appears you can extend the FM's capabilities pretty radically through the plugin API. It adds drag-to-move (rename) which is pretty fancy! (Of course Marlin has no rename or move file command, so maybe it is only working with local files?)
Do I want to make an "outline view" of the files and folders? Only if it's very simple! No, first I want to make a view that shows only the contents of 'current directory' including files and folders, with the standard interface to enter a folder, go up a level, and so on. I haven't seen that behavior, only the single flat list.
I will have to study that plugin and see if I can tweak it to get the desired behavior. One thing it needs to do is additionally parse the response from M20 and remember all unique paths so they can be shown as navigable folders in the FM, even if M20 doesn't include the folder names in the output. (But of course, I could also add folder names to the flat output of M20.)
The reason I'm initially adding "deep listing" to the Virtual Printer is to have better working data for developing the FM code, and it will also be a good ground to test out other work in progress, like the long-delayed compact JSON file listing. I'm making a tiny DOS (and debugger) interface for my own experimentation, but maybe that will become a viable feature in the future as well.
Yes! Currently when I use OctoPrint 1.6 to talk to a printer that has its own SD card, I do not see a nice interface like you are showing me. I'm glad to see that the capability exists, if only I can give the right hints to the File Manager.
I am starting to put in the necessary work, with my goal being to make the File Manager behave in the same manner whether you are interacting with the local media on the Raspberry Pi or media on the remote machine. Marlin now supports multiple drives (one active at a time) so it may also be useful to expose the available drives in the host interface.
The capability exists, because OctoPrint primarily focuses on locally stored files and there it supports this kind of stuff. SD card listings across firmware forks are all over the place, so so far we simply display those as-is, without splitting by folder. In principle it would be possible to do that, all that would be needed would be to adjust the returned data on the API, making sure though to generate the correct commands for file selection). However that would also require some thought to go into the UI. Right now we mix local and SD located files. That works because all SD files land in the root and the only folders come from local storage. If we want to create a folder hierarchy for SD files as well, the rendering needs to be changed.
We could go for a root with a virtual folder per file storage location. Or alternatively a virtual SDCARD or something like that folder in the otherwise local root.
Downside of the former is that if SD support is disabled in either OctoPrint or the printer, that would look incredibly weird. We'd also have to decide whether we want to default to showing root, local or sdcard in such a scenario.
The latter would possibly cause issues if people have a local folder called precisely the same. We could work around this by simply pinning other storage locations to the top separate from the usual folders.
Final idea: central switch over the file manager to select storage to look at. local, sdcard, maybe more in the future (USB sticks come to mind and that's actually something I've been looking into on the side for OctoPi).
I like this idea of a drop-down at the top to select the storage location. This would also allow for multiple printer items possibly with Marlin supporting multiple sd cards, etc.
Cheerio! Well, I can continue do things from the firmware side and keep exploring the OctoPrint code to find the best balance. I have this longstanding PR that I need to finish, which will allow Marlin to generate JSON file listings with M20 P and printer status reports with M408, both for the benefit of the PanelDue controller. I will initially just observe how OctoPrint handles these commands, and later I will work on improving how OctoPrint interacts with Marlin by building out the file dialog and assessing at that time what kinds of extensions I need to add to the firmware to make the whole experience as seamless as possible.
