Using ffmpeg for webcam streaming and timelapse support

[pmayer]

Well... it cost's me a complete evening to realise, that only the older C920's have h264/mp4 encoding. Logitech decided to drop this feature a while ago:

Article: Say Goodbye to In-Camera Hardware Encoding

According to Wikipedia, there are not many Webcam's with mp4 left:
https://en.wikipedia.org/wiki/List_of_cameras_with_onboard_video_compression

---

But nontheless I've found a way for me to make it work on my Raspberry Pi 4 running OctoPi 0.17.0 with OctoPrint 1.4.00: Restreamer (https://datarhei.github.io/restreamer/). Here's what I did:

• disabled/stopped webcamd and wait till port 8080 is free, check with netstat -tulpen

  sudo service webcamd stop
  sudo systemctl disable webcamd.service
  

• installed Restreamer via Docker (ARM version) on the Pi: https://datarhei.github.io/restreamer/docs/guides-usb-camera.html using port 8080 (e.g.) and /dev/video0
• installed @jneilliii's WebCamIframe-Plugin: https://community.octoprint.org/t/load-video-stream-instead-of-usb-camera/9772/19
• Started streaming via Restreamers Webinterface in h264 mode. If your Webcam supports native h264 you can simply use copy in the encoding options
• changed Octoprint's Stream-URL to: http://<network-ip-of-pi>:8080/player.html
• changed Octoprint's Snapshot-URL to: http://127.0.0.1:8080/images/live.jpg
• Configured and started restreamers Encoding on rtmp://127.0.0.1/live/usbcam.stream accordong to the guide.

I'm at 30-40% CPU with 720p@25fps (superfast, 4MBit/s) as I can't find anything on how to use the Pi's hardware mp4 unit for this.

But maybe some of you can help me out, as there is a second ffmpeg process eating up cpu time. I think this must be something from Octoprint itself. I don't know it this also blocks Port 8080, as I've disabled mjpeg_streamer by stopping webcamd.

Edit: Both ffmpeg processes are from Restreamer. One to fetch /dev/video0 and put it to /dev/video, the second to encode the video to h264.
Edit^2: Obviously I've not waited long enough to let free up mjpeg_streamer port 8080 after stopping it.

Before that I've fidled around with ffserver (https://trac.ffmpeg.org/wiki/ffserver), which is not maintained anymore. I've switched to Restreamer before I came to the conclusion, that my camera is lacking h264.

All in all it's kind of a mess. Every information you find regarding v4l2 is mostly from 2014/2015 and things like ffserver are discontinued. Don't get me wrong, I'm very thankfull for the work people put into this. But maybe I'm oversaw something as I went on the wrong path...

Would be great to get your input on my unsolved issues:

• Disabling Octoprint's ffmpeg
• Free up port 8080
• Employ hardware encoding on the Pi

As the hardware encoding bit has to do something with Restreamer in my setup, I've openend up an issue there:
https://github.com/datarhei/restreamer/issues/148 - solved Hardware encoding of the usbcam stream on the Pi. Now max 10% cpu load.
Keep in mind, restreamer always reencodes the webcam to h264, so on the settings page you only need to use "copy" to get mp4.

Cheers,
Patrik

(As a new user I'm only allowed to put two links in a post, thus the links in parenthesis)

[foosel]

@pmayer

Free up port 8080

Take a look at netstat -tulpen, it should point out the PID of the process that's listening on that (unprivileged) port, e.g.

$ netstat -tulpen | grep 8080
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      1000       10825210   5829/./mjpg_streame
tcp6       0      0 :::8080                 :::*                    LISTEN      1000       10825211   5829/./mjpg_streame

This tells us that it's PID 5829 listening on 8080 on both IPv4 and IPv6. It also tells us a truncated version of the process name here, but we can also cross check the PID:

$ ps -ef | grep 5829
pi        4923  4892  0 11:49 pts/0    00:00:00 grep --color=auto 5829
pi        5829  5801  0 Feb24 ?        00:20:39 ./mjpg_streamer -o output_http.so -w ./www-octopi -n -i input_uvc.so -r 640x480 -f 10 -d /dev/video1

[pmayer]

Hey @foosel,

thank you. Of course netstat should have come to my mind.
Oviously I've tried to fire up the docker container with restreamer while mjpeg_streamer was still running... or shutting down and the port was not free, yet.

Currently there is no process on 8080, so it should be fine. Thanks again.
And sorry, that I did not double check this before posting.

Cheers,
Patrik

[johnfreundracing]

GREAT solution, and thanks so much for sharing this. I couldn't believe I was watching a 5 fps low res jittery stream in 2020. Took me a bit to puzzle out your light instructions but I (like you I'm guessing) do this for a living so it wasn't that bad. On my Pi 4 I'm chugging along with a 1920x1080 30fps stream just a scratch to the cpus.

One odd thing though, the webcam tab video frame is quite small at least on my Octoprint page. Is that the case for you? If it was, did you by chance figure out a way to solve that? I haven't dug into code to see if there's some width and height vars I can change to display this better.

[johnfreundracing]

Replying to my own comment as I found a way to change the iframe size. It's in ~/.octoprint/generated/webassets/packed_plugins.js

Look for the line
$('#webcam_container').replaceWith('');

I set mine to 1920 x 1080 in that line, I think the original was 588 x 33x? I forget, something low. Being javascript you can just reload your your browser tab with octoprint and it will take effect immediately.

[pmayer]

Hey, great to hear. Yes, this was not meant to be a step-by-step guide, just wanted to share how I did it, as the documentation of restreamer is quite good.
If there's a demand I can maybe write this down in more detail.

@johnfreundracing

I couldn't believe I was watching a 5 fps low res jittery stream in 2020.

Same here... that's why I meant that the whole thing is kind of a mess

How much cpu load do you have at 1080p@30? Here all 4 cores are lifting between 30-60% for 720p. See htop for info.
Thus I'm seeking the hardware encodig ffmpeg. I've found an old NUC with a Pentium G5400... I will test about the cpu load on this.

Do you have an h264 capable camera?
How long is your delay? I have around 5-10 seconds.

Hmm, I think I did not have the size hassle, or do I?
But I admit, that I've also exposed a standalone player through nginx (https, letsencrypt) to only watch without the Octoprint controls.

[johnfreundracing]

Pi 4 is a four core device and load average sits at around 1.36, but no single CPU is maxed ever. At most I see Chromium taking up a good chunk of CPU but usually sub 50%, sometimes as high as 80. ffmpeg sits sub 10%. Looking at ffmpeg's parameters it's just do a copy rather than any transcoding so this doesn't really surprise me. My camera is a Raspberry Pi v2 camera which natively outputs H264, hence the transcoding not being necessary.

My delay is like 5-10 seconds. I hate that, I was trying to see if it's the player that's buffering it, or something in streaming. I mean it doesn't really matter, seeing the printer delayed doesn't really change the usefulness of this that much, but it does bug me

Your screen shot shows pretty low res to my eyes for that window. I am using a 1920x1080 stream, so I wanted my window to match. I'm using the Themify plugin to expand the interface making room for the Webcam iframe.

[pmayer]

@johnfreundracing

Looking at ffmpeg's parameters it's just do a copy rather than any transcoding so this doesn't really surprise me. My camera is a Raspberry Pi v2 camera which natively outputs H264, hence the transcoding not being necessary.

Ah, then it's clear that this is easy going.
I'm using a Logitech C920, sadly the new version without hardware encoding.
That's why I've opened an issue over at restreamers github as the Pi has hardware h264 encoding capabilities that would be great to use.

@johnfreundracing

My delay is like 5-10 seconds. I hate that, I was trying to see if it's the player that's buffering it, or something in streaming. I mean it doesn't really matter, seeing the printer delayed doesn't really change the usefulness of this that much, but it does bug me

I've found this regarding the delay, but did not implement it, yet:

Latency problems live stream is lagging 8-9 seconds

@johnfreundracing

Your screen shot shows pretty low res to my eyes for that window. I am using a 1920x1080 stream, so I wanted my window to match.

Well, I got an 1080p monitor and cause of the software encoding I thought 720p is enough for the video. Maybe I will bump it up once propper hardware encoding or the NUC is in place.

[OutsourcedGuru]

Rather than patching a generated file it might be a little safer to use Themeify and just make the edit there.

[johnfreundracing]

Oh yeah you're 100% right there. I have no idea (yet) how to make the tweak via Themeify but I suspected that would be the ultimate solution.

[johnfreundracing]

I tried copying the configs from the container and mounting them outside and editing the nginx.conf with the hls setting changes mentioned in that link about lag, and they didn't improve the lag at all for me. I'm staring at it wondering if I did something wrong.

[OutsourcedGuru]

Search the forum for "pimp" for a wide variety of edits we've all made.

[pmayer]

Well I'm no Docker person... I will look into this when I get feedback from the restreamer guys.

[johnfreundracing]

Per OutsourcedGuru's nudging about Themeify, it hit me that the settings were pretty simple to do:
Selector CSS-Rule Value
#webcam_container height 1080
#webcam_container width 1920

so no silly manually editing of the generated file which can get lost with other changes (happened to me which lit the fire under my butt to figure out how to do this in Themeify).

[Szafran]

@pmayer

installed @jneilliii's WebCamIframe-Plugin

Anyone knows if this works with MultiCam plugin ?

@pmayer

Well... it cost's me a complete evening to realise, that only the older C920's have h264/mp4 encoding

So now we need to use c930e only ?
I need to check if the c920 in my setup has h264 (it should because I've bought an older one second hand).

[pmayer]

@Szafran

Anyone knows if this works with MultiCam plugin ?

Puhh... I think @jneilliii would have to answer this.

@Szafran

So now we need to use c930e only ?

Not quite. I'm using the C920 but have some cpu load (40-50%) as it don't has native h.264, thus I can't use copy in the encoding options of restreamer. But it works.
Currently I know no way to use hardware h264-encoding which the Pi is cappable of. See my comment on the issue over at restreamers github:

ffmpeg hardware encoding on Rasbperry Pi

[jneilliii]

@Szafran

Anyone knows if this works with MultiCam plugin ?

Not really sure, never used multi cam, but my assumption is no because I think it just charges the src attribute of the image.

[Deses]

@pmayer

• changed Octoprint's Stream-URL to: http://<network-ip-of-pi>:8080/player.html

I'm having CORB issues when I use the 192.168.1.XXX ip of the Pi, and connection refused when I use 127.0.0.1... and I don't even know where do to begin to fix that.

I can see restreamer fine in my browser, though!

[jneilliii]

HLS support for OctoPrint....

Experimental HLS support

Hi! I have added experimental HLS support both to OctoPi (see devel branch) and OctoPrint. I think it's impossible to add it as a plugin, so it's will be a pull request to the main source. @fossel, please, tell if you think this way is a the proper way of adding HLS support.

[NeWFie-KaNaTa]

Setting up Octoprint on a old Toshiba laptop with Ubuntu 18.04. After a few days of struggle trying to get mjpg streamer to work I decided to try this and finally got a webcam feed... but the snapshot url isn't working. I can get more detail is needed any help would be awesome!!! (Next time I'm just buying the Pi, ha)