OctoPrint plugin that works with The Spaghetti Detective


#1

I built an OctoPrint plugin to works with The Spaghetti Detective (http://thespaghettidetective.com).

The plugin can be installed using this link: https://github.com/TheSpaghettiDetective/OctoPrint-TheSpaghettiDetective/archive/master.zip

The Spaghetti Detective is a Deep Learning-based failure detection server using nothing but a webcam. Just as OctoPrint itself, The Spaghetti Detective is also open source at https://github.com/TheSpaghettiDetective/TheSpaghettiDetective


#2

oooo, very interesting. I see you are listed on the team of TSD. I've been hoping something like this would come along.

I see the server can be run locally: https://github.com/TheSpaghettiDetective/TheSpaghettiDetective

Are you transcoding the webcam feed?


#3

Yup I'm part of the team.

The plugin still has some rough edges and it's currently just sending jpeg. We briefly tested it out and it looks like RPi is not powerful enough to transcode raw images to video. Will be happy to hear if there is a way to send webcam feed more efficiently.


#4

Can I see previous print attempts? Can I mark them as failed? (eg training)


#5

That's neat. Similar to the other plugin Project Kronos Data Collector

@tedder wouldn't this be a good use case for gstreamer since you mentioned it was more server to server? I know you're one of the few that I've seen experimenting with other video options, like the uv4l webrtc stuff.


#6

It is a good suggestion to use existing timelapses as training data. I think we can build this function in our next version!


#7

Just checked out gstreamer and it looks pretty cool. We previously tested transcoding with ffmpeg which was too CPU-demanding. Will give gstreamer a shot when we get a chance. Any other CPU-efficient transcoding lib you are aware of?

We saw project kronos and will be happy to collaborate with the author.


#8

Bonus points if you change the name of this to the FSM Detective.


#9

except that was sketchy as hell for about 100 reasons.

Possibly, but occasional jpegs seems good for this, because it doesn't need ongoing streaming.

Remember the Pi has hardware mp4(?) encoding, which is helpful. It was a really bad fit for me because I couldn't shove it into a browser, but it might be a really good fit for y'all.

I always have that sticker on my motorized vehicles. I need a tattoo of it but the Octocat will be first. And maybe the golang gopher..


#10

Yeah both my RTMPStreamer plugin and YouTubeLive plugin are just using ffmpeg inside a docker container. I agree sending images in this use case is probably perfectly fine. I'm not even sure that my ffmpeg commands are the most efficient or using the GPU h264 encoding. I need to dig deeper into that maybe something like this...?


#11

I didn't know Pi has a hardware mp4 encoder. Thank you so much for pointing this out.

Why couldn't you shove it into a browser?


#12

I never knew that you can run a docker inside RPi. Will have to steal some code from your plugins. :wink: Thank you in advance.


#13

HTML5 doesn't grok mp4 :confused:


#14

You should look at a port of my YouTube plugin. Allows for a lot more customization of the ffmpeg command line and is a smaller docker image overall I think.Not sure if the ffmpeg is compiled with hardware encoding support, but best I can tell from reading online, it's the h264_omx option you would want to use probably.

I think you could get this to work with a standard video tag. My MultiCamView view plugin (very undeveloped) is able to stream mp4. The specific video tag stuff....https://github.com/jneilliii/OctoPrint-MultiCamView/blob/83547419973bb0fa41b2e1b816d34b3dfd1cfdf7/octoprint_multicamview/templates/multicamview_tab.jinja2#L10


#15

Perfect! I'll definitely take a look!

If you have an option to add an extra javascript, I have used https://videojs.com/ and it works pretty nicely with mp4.