Issue
I'm experimenting with capturing and saving two video streams with USB cameras directly on a Raspberry Pi 4. The cameras appear in /dev
as /dev/video0
and /dev/video2
and both are tested independently to work with a frame rate of 30 fps.
I started both streams simultaneously with the command
sh -c 'ffmpeg -f v4l2 -r 25 -s 640x480 -i /dev/video2 left.avi & ffmpeg -f v4l2 -r 25 -s 640x48
0 -i /dev/video0 right.avi & wait'
I encountered that the frame rate of one camera is ~25fps while the other one shows the following output
frame=1 fps=0.2 q=2.9 size=6kB time=00:00:00.04 bitrate=1135.6kbits/s speed=0.00983x
One of the videos played as expected, in the other one just a few frames were stored. The same problem occurred when I tried to capture a video using OpenCV's VideoCapture()
class.
I've seen this question and reduced the frame rate to 15 in ffmpeg
like so
sh -c 'ffmpeg -f v4l2 -r 15 -s 640x480 -i /dev/video2 left.avi & ffmpeg -f v4l2 -r 15 -s 640x480 -i /dev/video0 right.avi & wait'
which didn't change the issue, just by looking at the file sizes of the two captured files we see that they differ extremely
ubuntu@ubuntu:~/code-py$ ls -l
total 520
-rw-rw-r-- 1 ubuntu ubuntu 490368 Mar 18 14:05 left.avi
-rw-rw-r-- 1 ubuntu ubuntu 5686 Mar 18 14:05 right.avi
and essentially the same happens using OpenCV's capture.set(cv2.CAP_PROP_FPS, 15)
.
In a third step I halved the video size like so
sh -c 'ffmpeg -f v4l2 -r 30 -s 320x240 -i /dev/video2 left.avi & ffmpeg -f v4l2 -r 30 -s 320x240 -i /dev/video0 right.avi & wait'
and it worked! Apparently something in the Raspberry gets overwhelmed by the amount of bytes flowing in, but how do I figure out what it is? All in all 320x240 is a pretty bad resolution and I wonder if it is possible to find the limiting factor and increase it.
Edit:
Here is the output of lsusb -t
with the cameras plugged into the USB 3 port:
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 3, If 0, Class=Video, Driver=uvcvideo, 480M
|__ Port 1: Dev 3, If 1, Class=Video, Driver=uvcvideo, 480M
|__ Port 1: Dev 3, If 2, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 1: Dev 3, If 3, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 2: Dev 4, If 0, Class=Video, Driver=uvcvideo, 480M
|__ Port 2: Dev 4, If 3, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 2: Dev 4, If 1, Class=Video, Driver=uvcvideo, 480M
|__ Port 2: Dev 4, If 2, Class=Audio, Driver=snd-usb-audio, 480M
Solution
An RGB888 stream of 640x480 data at 30fps takes the following bandwidth:
640 * 480 * 3 * 30 * 8 i.e. about 220 Mbits/s
And that is almost half the bandwidth of 480 Mbits/s USB 2 so you are not realistically going to be able to run 2 such streams smoothly.
You can check USB bandwidth using the lsusb -t
command. Here is a memory stick plugged into the USB 2 port - you can see it is allocated 480 Mbits/s:
lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 3: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 480M <--- HERE
And here is the device plugged into the blue USB 3 port nearer to the Ethernet - allocated 5Gbits/s:
lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
|__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M <--- HERE
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
Keywords: USB 2, USB 3, bandwidth, speed allocation.
Answered By - Mark Setchell Answer Checked By - Gilberto Lyons (WPSolving Admin)