Webcam Randomly Stops Working

agor · December 10, 2020, 3:45pm

Still looking into it guys, i am not giving up.
l'll let you know if i nail it.

For the time being:
I taped the power pin in the usb cable that connects my Ender5 to the pi, no effect.
I booted with the camera disconnected.
Plugged the camera after everything started.
Manually started webcamd.
It worked flawlessly for more than 6 hours while a print was running...

However:
Today the camera is back to its previous behaviour no matter what i do.
Manually stopping/restarting webcamd, killing mjpg_streamer, remove/reinsert uvcvideo did not have the desired effect.

When i reach a consistent behaviour i'll let you know.

agor · December 10, 2020, 3:47pm

Follow up:
If it means anything, after the camera "crash" the /dev/video0 is gone...

agor · December 10, 2020, 4:08pm

I believe i solved it, no hiccups so far.
It's not kernel related see my previous post.
It has to do with power suspending on usb devices.

Note to bzowk: This is not a pi4 issue, I am using a pi3b+.
I can't say if it applies to a pi4...

If i post everything it's going to be a very long post, i'll try to make it short:

I have a fan permanently connected to the pi, first thing i did was disconnecting it in case this is power related.
Problem still there***.
I added a udev rule to disable the microphone.
Problem still there***.

manually start webcamd***

tail -f /var/log/syslog

after a few seconds i got a "usb disconnected"***

lsusb

what is of interest is:
Bus 001 Device 006: ID 045e:0779 Microsoft Corp. LifeCam HD-3000
so camera still there

ls /dev/video*

/dev/video0 gone

So i figured this has to do with suspending.
created a udev rule specific to camera autosuspend:

nano /etc/udev/rules.d/90-disable-lifecam-autosuspend.rules

add the line (note the previous output from lsusb and adjust accordingly):
ACTION=="add", SUBSYSTEM=="usb", ATTRS{idVendor}=="045e", ATTR{idProduct}=="0779", ATTR{product}=="Microsoft Corp. LifeCam HD-3000", TEST=="power/control", ATTR{power/control}:="on"

If anybody is interested i am happy to post all the details (marked with ***).

I am not sure if this is the "safe" way of solving it, please let me know if it is not.

kuishi · December 11, 2020, 9:33am

Just confirming I have the same issue. Raspi4B, Default Ribbon Cable Cam.

As long as I was using version 1.4.x, everything worked fine. Once I updated to 1.5.x, the camera works initially but randomly becomes unavailable, and only Raspi system reboot can reactivate it. Reboot works reliably to get cam fixed however so unlikely this is a simple ribbon cable issue.

My plugins are:

Consolidated Tabs (0.1.3)
Cost Estimation (3.1.0)
DisplayLayerProgress Plugin (1.24.0)
EEPROM Editor for MPSM (0.1.1)
Filament Manager (1.6.3)
Malyan/Monoprice Connection Fix (0.1.3)
Marlin GCode Documentation (0.11.0)
Octoprint-Display-ETA (1.1.1)
PrettyGCode (1.2.3)
PrintTimeGenius Plugin (2.2.7)
Sticky Pad (0.1.4)
TouchUI (0.3.17)
TP-Link Smartplug (0.9.26)

Vanilla MJPEG says this once the cam fails:
HTTPConnectionPool(host='127.0.0.1', port=8080): Read timed out. (read timeout=5)

Since it worked so well before and so reliably unreliable after the update, I suspect a connection to the update.

Not sure if it would help if I would post any logs... which ones would?

kuishi · December 11, 2020, 9:36am

Addendum:

sudo service webcamd restart

seems to help when the issue occurs... but I'd rather not have the fix be scripting something that SSH spams this command to my octopi

kuishi · December 11, 2020, 10:05am

This is my webcamd working normally (from journalctl -u webcamd):

Dec 11 01:47:14 octopi webcamd[341]: Starting up webcamDaemon...
Dec 11 01:47:14 octopi webcamd[341]: --- Configuration: ----------------------------
Dec 11 01:47:14 octopi webcamd[341]: cfg_file: /boot/octopi.txt
Dec 11 01:47:14 octopi webcamd[341]: camera: auto
Dec 11 01:47:14 octopi webcamd[341]: usb options: -r 640x480 -f 10
Dec 11 01:47:14 octopi webcamd[341]: raspi options: -fps 10
Dec 11 01:47:14 octopi webcamd[341]: http options: -w ./www-octopi -n
Dec 11 01:47:14 octopi webcamd[341]: Explicitly USB device:
Dec 11 01:47:14 octopi webcamd[341]: -----------------------------------------------
Dec 11 01:47:14 octopi webcamd[341]: Found video devices:
Dec 11 01:47:14 octopi webcamd[341]: /dev/video0
Dec 11 01:47:14 octopi webcamd[341]: /dev/video10
Dec 11 01:47:14 octopi webcamd[341]: /dev/video11
Dec 11 01:47:14 octopi webcamd[341]: /dev/video12
Dec 11 01:47:14 octopi webcamd[341]: raspi
Dec 11 01:47:14 octopi webcamd[341]: config file='/boot/octopi.txt':USB device was not set in options, start MJPG-streamer with the first found video device: /dev/video0
Dec 11 01:47:14 octopi root[419]: Starting USB webcam
Dec 11 01:47:14 octopi webcamd[341]: <13>Dec 11 01:47:14 root: Starting USB webcam
Dec 11 01:47:14 octopi webcamd[341]: Running ./mjpg_streamer -o output_http.so -w ./www-octopi -n -i input_uvc.so -r 640x480 -f 10 -d /dev/video0
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: starting application
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: MJPG Streamer Version: git rev: 501f6362c5afddcfb41055f97ae484252c85c912
Dec 11 01:47:14 octopi webcamd[341]: MJPG Streamer Version: git rev: 501f6362c5afddcfb41055f97ae484252c85c912
Dec 11 01:47:14 octopi webcamd[341]: i: Using V4L2 device.: /dev/video0
Dec 11 01:47:14 octopi webcamd[341]: i: Desired Resolution: 640 x 480
Dec 11 01:47:14 octopi webcamd[341]: i: Frames Per Second.: 10
Dec 11 01:47:14 octopi webcamd[341]: i: Format............: JPEG
Dec 11 01:47:14 octopi webcamd[341]: i: TV-Norm...........: DEFAULT
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: Using V4L2 device.: /dev/video0
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: Desired Resolution: 640 x 480
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: Frames Per Second.: 10
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: Format............: JPEG
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: TV-Norm...........: DEFAULT
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Pan (relative): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Tilt (relative): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Pan Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Tilt Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_ADD - Error at Focus (absolute): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Pan (relative): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Tilt (relative): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Pan Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Tilt Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Focus (absolute): Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at LED1 Mode: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at LED1 Frequency: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Disable video processing: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: UVCIOC_CTRL_MAP - Error at Raw bits per pixel: Inappropriate ioctl for device (25)
Dec 11 01:47:14 octopi webcamd[341]: o: www-folder-path......: ./www-octopi/
Dec 11 01:47:14 octopi webcamd[341]: o: HTTP TCP port........: 8080
Dec 11 01:47:14 octopi webcamd[341]: o: HTTP Listen Address..: (null)
Dec 11 01:47:14 octopi webcamd[341]: o: username:password....: disabled
Dec 11 01:47:14 octopi webcamd[341]: o: commands.............: disabled
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: www-folder-path......: ./www-octopi/
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: HTTP TCP port........: 8080
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: HTTP Listen Address..: (null)
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: username:password....: disabled
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: commands.............: disabled
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: starting input plugin input_uvc.so
Dec 11 01:47:14 octopi mjpg_streamer[420]: MJPG-streamer [420]: starting output plugin: output_http.so (ID: 00)
Dec 11 01:47:15 octopi webcamd[341]: Done bring up all configured video device
Dec 11 01:47:15 octopi webcamd[341]: Goodbye...
Dec 11 01:47:15 octopi systemd[1]: Started the OctoPi webcam daemon with the user specified config.

This is where shit hit the fan (not sure if there is a way I can get this to log more verbosely?):

Dec 11 08:07:05 octopi webcamd[341]: i: select() timeout
Dec 11 08:07:05 octopi webcamd[341]: i: cleaning up resources allocated by input thread
Dec 11 08:07:05 octopi mjpg_streamer[420]: MJPG-streamer [420]: select() timeout
Dec 11 08:07:05 octopi mjpg_streamer[420]: MJPG-streamer [420]: cleaning up resources allocated by input thread

This happenend after I ran "sudo service webcamd restart":

Dec 11 10:34:27 octopi systemd[1]: Stopping the OctoPi webcam daemon with the user specified config...
Dec 11 10:34:27 octopi systemd[1]: webcamd.service: Main process exited, code=killed, status=15/TERM
Dec 11 10:34:27 octopi systemd[1]: webcamd.service: Succeeded.
Dec 11 10:34:27 octopi systemd[1]: Stopped the OctoPi webcam daemon with the user specified config.
Dec 11 10:34:27 octopi systemd[1]: Starting the OctoPi webcam daemon with the user specified config...

The rest sounds normal, and it's been working since.

So the more permanent solution was running the following three steps in SSH:

sudo apt-get update --fix-missing
sudo apt-get upgrade
sudo reboot

As @foosel correctly pointed out, it's not Octoprint's fault if the Octopi enviroment it is running on becomes outdated.

So unless you frequently re-etch the latest Octopi to your SD card anyways, this is probably one of the default maintenance actions you have to do once in a while to prevent Octopi from eventually going downhill...

Again maybe at some point we can get an additional reboot option to execute these steps into octoprint's top bar (in addition to reboot system, shutdown system, there could be a new "udpate and reboot system" option doing effectively the above).

LordDarkneo · December 11, 2020, 10:28pm

Hi and thanks for the command on webcamd restart (working even when printing, not impacting the print)

But I am quite surprised with the sudo command (that I performed for sure few days ago, when trying the 1.5.0 rc3). I will execute them again, but even my octopi is 0.17 (the lastest version)... I do not think this will fix the issue efficiently, at least not in my case....

kuishi · December 12, 2020, 6:41am

My original suspicion was that with 1.5.X something changed about the way octoprint talks to webcamd / MJPEG that sometimes makes those crash.

But I don't seem to understand enough about the process to effectively debug it, and it sounds like the devs are saying in other parts of this forum that no such change has been made. Power supply shortage was suspected at some point as well, in addition to mechanical issues (e.g. in the ribbon cables), but I could rule those out.

For me the apt-get upgrade seems to have reliably solved the issue, but i guess in other cases scripting something that regularly makes webcamd restart (like 5 min intervals or so) might be the best bet for a workaround until someone gets to the root of the issue. As you said, restarting webcamd over SSH does NOT mess with ongoing prints or any other vital operation so should be safe even if done pretty regularly.

Charlie_Powell · December 12, 2020, 9:48am

The only change that was made anywhere near the webcam embedding was a quick permissions check of whether it should be loaded - and note that this is purely on the embedding side, and once the web UI is loaded it is either there, visible or it is not. And this was actually a bug fix, since it should have been there anyway.

The way embedding MJPG streams works, is with an <img> tag, set the src to the webcam stream and away we go. OctoPrint doesn't even know MGPG streamer exists, let alone how to mess around with it.

With every update, there seems to be cases of system instability that cause problems, like the SerialException thread, nothing was changed that could case this, but the upgrade somehow made people's printers disconnect more often. No idea whether the few isolated cases of MJPG streamer dying could be related, or whether it is coincidence - what tends to happen is that around updates, the people who would have usually run into problems think the problem was the update. Same number of people in a months time, but no update to blame.

LordDarkneo · December 22, 2020, 9:16am

Hi there,
Just another update on this topic. I also find out another way to fix (temporarly) the issue:

sudo pkill mjpg_streamer

Excute this command twice will also make the webcam working again....

agor · December 25, 2020, 10:53am

Hi there, merry christmas to all.

An idea for a workaround could be a button in the control interface to restart the camera service:

sudo service webcamd stop
sudo pkill mjpg-streamer
sudo rmmod uvcvideo
sudo modprobe uvcviceo
sudo service webcamd start

Or even better, tracking the service and if it fails run the above commands.

Either way this is only a workaround, it can't be considered a solution...

Steve_Belt · January 5, 2021, 8:56pm

I was suffering from this issue, and found that doing a raspbian OS upgrade has fixed my problem:

sudo apt-get update --fix-missing
sudo apt-get upgrade
sudo reboot

Although I have a rather new installation (less than 30 days ago), my kernel was 2019-09 vintage. The latest kernel is 2020-12.

agor · January 23, 2021, 9:02am

@Steve_Belt thanks, it seems to have solved it (so far).
Although I just updated to 0.18.rc2, i can't say if is related to that.
The first thing i did was to upgrade everything, then i connected to the web interface.
I haven't run into issues yet...

I'll check it thoroughly through the day and post back later.

Cheers!

agor · January 24, 2021, 12:03pm

the problem is still there.
yesterday it was fine, today i booted the pi, no camera, reboot it was working.
but it stopped when the print started.
at least that's a hint...

i'll start messing with configuration files after the print is finished.

agor · January 24, 2021, 12:08pm

Update: adding -y in octopi.txt seems to have an effect so far.

Update:
Nope, it was only temporary, it persists...
I have some theories. If i come up with something definite or a solution i'll post back.

Sorry for the mess...

EDIT:

Well, i am at 0.18 now, the rc2 lasted two days...

The camera issue was still there. However, a change in /boot/octopi.txt seems to have solved it.

Changing:
camera_usb_options="-r HD -f 30"
to:
camera_usb_options="-r HD -f 25"

seems to behave consistently consistent for the last two days....

My guess is it has to be related to my Microsoft Lifecam linked with mjpg_streamer's implementation.

I did a lot of searching to reach to this conclusion.

I'll have to do thorough crash-testing, bu tif you people have issues, try it with your setup and let me know what happens, i am interested in the findings.

Cheers to everybody.

EDIT:

Go figure...
I tried the camera on my pc, it behaved exactly the same, although it wasn't doing it some months ago...
So it definitely is not even pi related.
I tried with my girlfriends camera - a cheap one - no issues.

I did two things.
First, the usb plug looked rusty to my eyes, so i put in wd40 for about 10-15 minutes and carefully dried it.
Then i very carefully tightened the plug with a plier, the connection felt sort of loose.

It worked fine on the pc for about 10 minutes, then i unplugged it and plugged it to the pi.
It started without the need for an ssh connection, no commandline tinkering at all.
For about an hour now i have no problems....

Weird, i never thought of that...

EDIT:

Go figure...
I tried the camera on my pc, it behaved exactly the same, although it wasn't doing it some months ago...
So it definitely is not even pi related.
I tried with my girlfriends camera - a cheap one - no issues.

I did two things.
First, the usb plug looked rusty to my eyes, so i put it in wd40 for about 10-15 minutes and carefully dried it.
Then i very carefully tightened the plug with pliers, the connection felt sort of loose.

It worked fine on the pc for about 10 minutes, then i unplugged it and plugged it to the pi.
It started without the need for an ssh connection, no commandline tinkering at all.
For about an hour now i have no problems....

Weird...

UPDATE:

Next day, still no issues, i am printing something that is close to1.5 days with one pause so far, not finished yet.

But i still cannot accept that it was only a hardware issue...

UPDATE#2: About 4 hours later, still no hiccups.

UPDATE#3: The Problem came back.
In my frustration i disassembled the camera and saw that a soldering was dodgy (white wire).
Fixed it...
Attaching an image after the repair.

I apologize for all the mess, it was a hardware issue after all...

LordDarkneo · May 6, 2021, 7:03am

Hello there,

After being fed up with all those webcam issues, I decided to have a deeper look on Internet. As already said in this thread, nor Octopi or Octoprint is delivering webcam drivers or mjpg_streamer components. The issue is really on Debian side. So, first thing first, let's see if there is any topic related to my webcam on RPI... And I found this tutorial:

I only followed steps 2-3-4, and I was surpised that the guvcview was not installed on my RPI.

BTW, after rebooting, the webcam is streaming fine for the moment. I'll keep you posted if it definitely solves this %*$^§!ng problem

crashoverride68 · May 19, 2021, 12:02am

I have two printers at home, and two printers at a makerspace. What has been an interesting case study, is that they all run the exact same OS, edition of octopi, hardware (3B+), and Webcam. 3/4 of them are even MK3s printers.

Two webcam setups at my home are fine, and work 24/7 for months now. At the makerspace #1 is dead, and #2 works intermittently (Will; run for 2-6 days before stopping and requiring a restart).

As best as I can tell, my camera issues stem from a hardware issue. Here's my attempt at a troubleshooting guide brought together from all of the stuff throughout this thread.

Update your OS

sudo apt-get update --fix-missing
sudo apt-get upgrade
sudo reboot

Try a different Webcam/cable.
Try your current webcam on another device to verify it's working for ~12 hours.

There are a bunch of other fixes via software you can try after those, and can be found above. I don't want to list them here, since most of them are hardware specific. In my case, 2/4 of the webcams were dead/dying. My lesson I learned from this thread is that cheaper, or even not so cheap webcams can and will go bad.

Thank you to @agor & @Steve_Belt and everyone else keeping this thread going for so long.

agor · July 7, 2021, 9:05pm

Hi everybody and crashoverride68

More than 5 months after the camera fix, zero issues.

As far as i am concerned the problem is solved.

Best wishes to all.

igor · December 17, 2021, 11:26pm

Hi @agor , I ended up here experiencing similar problems. I found a different cause and workaround (RPI4) and I wonder if by any chance the same happened to you, unnoticed.
In my experience, if the frame changes a lot, for exemple I move my hand rapidly in front of the cam, the systems craps out, to the point that /dev/video0 disappears. This happens reliably with the cam connected to a USB2 port, but if it's connected to a USB3 port, then everything is fine. I'm running Ubuntu 21.10
Could it be that initially you had the cam on a slow USB2 port and then while testing it got moved to the faster USB3?

agor · December 19, 2021, 9:59am

Hi @igor
Kind of sounds funny, igor asking agor...

Anyways, i am also a linux guy, for the last couple of years i'm using arch.
I am using a RPi3 B+ to control my printer, it has 4 usb2 ports.
I haven't noticed the camera giving up on me in similar cases.
For instance when i calibrate the bed with the camera showing the feed, and i move my hands and the printhead around, the camera did not give up.
So i can't really say how an RPi4 behaves...

I have an issue lately with firefox with freezes when i switch to the camera tab.
I am not sure if this is because i updated to the latest firefox or because i changed the settings to have it load exclusively to ram, trying to minimize disk writes on my nvme - maybe its overkill...
Chromium behaves nicely though, so i haven't looked deeper into it.