Since you've added nginx I don't know if it's necessary to visit the haproxy configuration on there to adjust anything but perhaps not. As long as they're not fighting over ports, it's probably alright.
1 Like
Great write up. I wonder why @guysoft doesn't use ffmpegserver in lieu of mjpgstreamer in octopi. My best guess would be because of some issue with camera support maybe?
Which codebase is that?
Welcome to contribute info to this issue:
yes, ffmpeg is compiled on the raspberry pi but BUSTER OS is installed from the raspberry pi website
1 Like
yes, FFMPEG was compiled on BUSTER
Can someone give me a straightforward guide how to add this to OctoPi?
I do not know how you create the Octopi image. The latest version of FFMPEG that octopi uses DOES NOT contain the FFSERVER. That is why I point you to downloading version 3.4.6
This version (3.4.6) is maintained by someone who wanted to keep ffserver around to be used. I would assume that since ffmpeg is already installed with Octoprint, that creating a separate directory for this version to live in and then rename the ffmpeg version of the exe to ffmpeg3.4.6 that you could allow both versions of ffmpeg to live on the octopi image.
So download the ffmpeg v 3.4.6, follow my instruction above for compiling it on raspberry pi, then grab the ffmpeg.exe rename to ffmpeg3.4.6 and grab the ffserver.exe image and place them into a directory on your octopi image. Make PATH aware of the location. I know I have over simplified this but I do not create images for raspberry pi but I have worked as a instrumentation engineer.
BWT, the FFMPEG format for rtsp, is something I am still adjusting. I am finding out that their are different rtsp streams, some transmit via UDP protocol others use TCP, and yet others use UDP for control messages and TCP for the video. I recently have found that the FFMPEG format is rtmp and I can see it using VLC media player, but OMXPLAYER can not play the stream, OMXPLAYER expects all video streams to be rtsp over tcp. So I have to figure out how to force FFMPEG to send the stream rtsp over tcp. With the setup here, it is using rtsp over UDP (or the mix of UDP and TCP). If someone knows how to force FFMPEG to do rtsp over tcp, please post the solution here.
No need for you to build the image (though you are always welcome to learn, I can help with that).
What I do need is a step-by-step guide. What you wrote above looks like a good start, I don't think I saw it earlier.
If you have a single scipt that does it, its even better.
Do you want the script to run under Buster OS? So, I understand your need, you want a bash script that will download ffmpegv3.4.6 and then compile this version of ffmpeg, rename the compiled ffmpeg to ffmpeg3.4.6 and change the PATH variable to point to this version of ffmpeg and ffserver? Is that what you need?
Yes
Yes
If they have a make install I'd go with what it does, you can also use checkinstall
If you are unsure how to do this just place it anywhere for now, And I will finalize it.
I dont think we should have the version name in the folder, because it means it will constantly change.
my concern is I do not want to over ride the ffmpeg version that automatically gets installed with the BUSTER OS by default, so how do you want to handle that situation? Or maybe it does not matter because octoprint.img is built with which OS?
Give me about a week to try to get a script written, OK?
Sure! No rush 
You can place vlc in /usr/local or /opt That would not get overridden
Usually in a case like this, a symlink in the right location can allow you to have multiple versions of the same software installed on a system. Run...
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
...and...
sudo sh
echo $PATH
/root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
exit
So then you could create two symlinks in /root/bin and /usr/local/sbin, for example, which could select the one you really want. But then when referring to that program name later you'd want to probably remove the path prepending that.
@guysoft I've did a lot of testing yesterday and have come to the opinion, that replacing mjpg_streamer with FFmpeg and FFserver and streaming the webcam feed with that is not a good idea right now, because performance is actually not that great. Remember: The example above uses a resolution of 480x360 @ 10 fps which causes a CPU load of 19% on a RPi4. FFmpeg would probably kill performance on anything oder than that.
I followed the great tutorial by @GadgetAngel with my Pi4 (Did a manual setup of Raspbian, Octoprint and followed the steps above, but used HAProxy) and fiddled around with the settings, but it seams like FFmpeg refuses to just copy the MJPEG stream that comes from the webcam and reencodes it instead.
The terminal output of the ffmpeg commands shows:
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 68514.700224, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 20 fps, 20 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, ffm, to 'http://localhost:8080/camera.ffm':
Metadata:
creation_time : now
encoder : Lavf57.83.100
Stream #0:0: Video: mjpeg, yuvj422p(pc), 1280x720, q=3-3, 15360 kb/s, 20 fps, 1000k tbn, 20 tbc
Metadata:
encoder : Lavc57.107.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/15360000 buffer size: 0 vbv_delay: -1
instead of:
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Nevertheless I ran some performance tests with the Pi4 and my Logitech C930e.
| Resolution | Framerate | Format | Threads | CPU Usage |
|---|---|---|---|---|
| 480x270 | 10 | MJPEG -> MJPEG | 1 | 25% |
| 1280x720 | 17 (15 from Cam) | MJPEG -> MJPEG | 1 | 100% |
| 1280x720 | 20 | MJPEG -> MJPEG | 2 | 70% + 40% |
| 1280x720 | 24 | MJPEG -> MJPEG | 2 | 80% + 50% |
Then I crammed out my Pi3 with a default OctoPi installation, and did some comparison with mjpeg_streamer:
| Resolution | Framerate | Format | CPU Usage |
|---|---|---|---|
| 960x540 | 10 | MJPEG -> MJPEG | 0.7% |
| 1280x720 | 20 | MJPEG -> MJPEG | 2% |
As you can see, the performance impact of mjpeg_streamer on a Pi3 is negligible while the Pi4 with FFmpeg could only push out 1280x720 at a maximum of 17 FPS when using only one thread. On the other hand I've noticed that the required bandwith for 20 FPS is less than half when using FFmpeg. (6 MBit/s vs 16 MBit/s)
Because the Pi4 is just so fast I've settled on 720p@20 FPS with 2 Threads for now as the decreased bandwith from reencoding is actually more beneficial for my setup. Printing works fine and the OctoPrint interface still shows up a LOT quicker than with a Pi3. (Config in this gist)
Where I think FFmpeg gets really interesting is streaming H264 encoded video, though this comes with it's own intricacies, as you can't just push out a mp4 file for a live stream and latency as well as protocol support is tricky. (RTSP, RTMP is not supported in browsers)
I think one has the best chances with HLS (HTTP Live Streaming) as Android as well as iOS and Edge + Safari support it natively and all other desktop browsers can be made compatible with a little bit of Javascript.
I thought I could use the H264 output of my C930e directly, but that is unfortunately not supported in the linux kernel right now, so I've tested with the hardware accelerated h264_omx codec and was able to achieve:
| Resolution | Framerate | Format | Threads | CPU Usage |
|---|---|---|---|---|
| 1280x720 | 24 | MJPEG -> H264 (OMX-Encoder) | 1 | 95% |
Most CPU usage I guess comes from decoding the MJPEG stream of the webcam. Latency was about 1 second. (Example Config in this gist)
The only problem I currently see is software support in anything that provides access to OctoPrint. (Apps like OctoRemote and Printoid or slicer integration as in Cura) Support for H264 is nonexistent, as they seem to be written to strictly expect an MJPEG stream.
2 Likes
Great thread tyvm! I'm new here and debating on where to start which of my Pi's to use for what and if best to stick with the 3b+ for now or pop in my 4 i just got. Honestly after reading this I may try both, I have 2 3d printers atm and this was a great intro thread to this message board in general for me.
Thanks again,
Shawn
I'll just throw this out there... I got interested in streaming v264/5 directly to the browser (Chrome), and cobbled together something that worked well 720p@30fps, smooth as butter, single digit CPU. This was with v4l and some python scripts I found. It was a proof of concept -- if it can be displayed itself in the browser, it can be displayed in a window on a web page.
I'll see if I can put it together again and share, maybe it will help. In any case, this is how streaming should be implemented in Octoprint.
If it works directly in a browser, you can use my WebcamIframe plugin to replace the image on the Control tab with an iframe to the URL you provide for the stream.