I've implemented some file browsers in my plugin, and noticed a really bad memory issue. I'm sending some rather large files to the client, and no matter what I do it consumes a terrible amount of memory, crashing my PI pretty fast. I've tried streaming the files using a generator + stream_with_context. I've tried send_from_directory. No dice. Htop even crashed while I was monitoring the usage, so it's filling the memory up 100%.
My working theory is that Octoprint is working on the request behind the scenes, and putting them into memory. I looked at the LargeResponseHandler, but it would be complicated to use since I'm dynamically creating some of the data. Perhaps it could be modified somehow? I also need to be able to delete files after the request is finished.
Any ideas? I'm desperate for a solution, because a ton of functionality I've added is rendered useless if I can't download a few 100mb files in a row without crashing the server.