Reading and Writing Videos using OpenCV
Reading and writing videos in OpenCV is very similar to reading and writing images. A video is nothing but a series of images that are often referred to as frames. So, all you need to do is loop over all the frames in a video sequence, and then process one frame at a time. In this post, we will demonstrate how to read, display and write videos from a file, an image sequence and a webcam. We will also look into some of the errors that might occur in the process, and help understand how to resolve them.
Let’s go through the code example for reading a video file first.This essentially contains the functions for reading a video from disk and displaying it. As you proceed further, we will discuss the functions in detail used in this implementation.
Python
import cv2
# Create a video capture object, in this case we are reading the video from a file
vid_capture = cv2.VideoCapture('Resources/Cars.mp4')
if (vid_capture.isOpened() == False):
print("Error opening the video file")
# Read fps and frame count
else:
# Get frame rate information
# You can replace 5 with CAP_PROP_FPS as well, they are enumerations
fps = vid_capture.get(5)
print('Frames per second : ', fps,'FPS')
# Get frame count
# You can replace 7 with CAP_PROP_FRAME_COUNT as well, they are enumerations
frame_count = vid_capture.get(7)
print('Frame count : ', frame_count)
while(vid_capture.isOpened()):
# vid_capture.read() methods returns a tuple, first element is a bool
# and the second is frame
ret, frame = vid_capture.read()
if ret == True:
cv2.imshow('Frame',frame)
# 20 is in milliseconds, try to increase the value, say 50 and observe
key = cv2.waitKey(20)
if key == ord('q'):
break
else:
break
# Release the video capture object
vid_capture.release()
cv2.destroyAllWindows()
C++
// Include Libraries
#include<opencv2/opencv.hpp>
#include<iostream>
// Namespace to nullify use of cv::function(); syntax
using namespace std;
using namespace cv;
int main()
{
// initialize a video capture object
VideoCapture vid_capture("Resources/Cars.mp4");
// Print error message if the stream is invalid
if (!vid_capture.isOpened())
{
cout << "Error opening video stream or file" << endl;
}
else
{
// Obtain fps and frame count by get() method and print
// You can replace 5 with CAP_PROP_FPS as well, they are enumerations
int fps = vid_capture.get(5);
cout << "Frames per second :" << fps;
// Obtain frame_count using opencv built in frame count reading method
// You can replace 7 with CAP_PROP_FRAME_COUNT as well, they are enumerations
int frame_count = vid_capture.get(7);
cout << " Frame count :" << frame_count;
}
// Read the frames to the last frame
while (vid_capture.isOpened())
{
// Initialise frame matrix
Mat frame;
// Initialize a boolean to check if frames are there or not
bool isSuccess = vid_capture.read(frame);
// If frames are present, show it
if(isSuccess == true)
{
//display frames
imshow("Frame", frame);
}
// If frames are not there, close it
if (isSuccess == false)
{
cout << "Video camera is disconnected" << endl;
break;
}
//wait 20 ms between successive frames and break the loop if key q is pressed
int key = waitKey(20);
if (key == 'q')
{
cout << "q key is pressed by the user. Stopping the video" << endl;
break;
}
}
// Release the video capture object
vid_capture.release();
destroyAllWindows();
return 0;
}
These are the main functions in OpenCV video I/O that we are going to discuss in this blog post:
cv2.VideoCapture– Creates a video capture object, which would help stream or display the video.cv2.VideoWriter– Saves the output video to a directory.- In addition, we also discuss other needed functions such as
cv2.imshow(),cv2.waitKey()and theget()method which is used to read the video metadata such as frame height, width, fps etc.
In this example, you will be reading the above video (‘Cars.mp4’) and displaying it.
We have provided the codes both in Python and C++ languages for your study and practice.
Now let’s start with it.
First we import the OpenCV library. Please note that for C++, you would normally use cv::function(), but because we choose to use the cv namespace (using namespace cv), we can access the OpenCV functions directly, without pre-pending cv:: to the function name.
Python
# Import libraries
import cv2
C++
// Include Libraries
#include<opencv2/opencv.hpp>
#include<iostream>
using namespace std;
using namespace cv;
Reading Video From a File
The next code block below uses the VideoCapture() class to create a VideoCapture object, which we will then use to read the video file. The syntax for using this class is shown below:
VideoCapture(path, apiPreference)
The first argument is the filename/path to the video file. The second is an optional argument, indicating an API preference. Some of the options associated with this optional argument will be discussed further below. To learn more about apiPreference, visit the official documentation link VideoCaptureAPIs.
Python
# Create a video capture object, in this case we are reading the video from a file
vid_capture = cv2.VideoCapture('Resources/Cars.mp4')
C++
# Create a video capture object, in this case we are reading the video from a file
VideoCapture vid_capture("Resources/Cars.mp4");
For all the examples given below, you can use the same video file that we have used here, which is available in the Downloadable Code folder, or you can use your own video file.
Now that we have a video capture object we can use the isOpened() method to confirm the video file was opened successfully. The isOpened() method returns a boolean that indicates whether or not the video stream is valid. Otherwise you will get an error message. The error message can imply many things. One of them is that the entire video is corrupted, or some frames are corrupted. Assuming the video file was opened successfully, we can use the get() method to retrieve important metadata associated with the video stream. Note that this method does not apply to web cameras. The get() method takes a single argument from an enumerated list of options documented here. In the example below, we provided the numeric values 5 and 7, which correspond to the frame rate (CAP_PROP_FPS) and frame count (CAP_PROP_FRAME_COUNT). The numeric value or the name can be supplied.
Python
if (vid_capture.isOpened() == False):
print("Error opening the video file")
else:
# Get frame rate information
fps = int(vid_capture.get(5))
print("Frame Rate : ",fps,"frames per second")
# Get frame count
frame_count = vid_capture.get(7)
print("Frame count : ", frame_count)
C++
if (!vid_capture.isOpened())
{
cout << "Error opening video stream or file" << endl;
}
else
{
// Obtain fps and frame count by get() method and print
int fps = vid_capture.get(5):
cout << "Frames per second :" << fps;
frame_count = vid_capture.get(7);
cout << "Frame count :" << frame_count;
}
After retrieving the desired metadata associated with the video file, we are now ready to read each image frame from the file. This is accomplished by creating a loop and reading one frame at a time from the video stream using the vid_capture.read() method.
The vid_capture.read() method returns a tuple, where the first element is a boolean and the next element is the actual video frame. When the first element is True, it indicates the video stream contains a frame to read.
If there is a frame to read, you can then use imshow() to display the current frame in a window, otherwise exit the loop. Notice that you also use the waitKey() function to pause for 20ms between video frames. Calling the waitKey() function lets you monitor the keyboard for user input. In this case, for example, if the user presses the ‘q’ key, you exit the loop.
Python
while(vid_capture.isOpened()):
# vCapture.read() methods returns a tuple, first element is a bool
# and the second is frame
ret, frame = vid_capture.read()
if ret == True:
cv2.imshow('Frame',frame)
k = cv2.waitKey(20)
# 113 is ASCII code for q key
if k == 113:
break
else:
break
C++
while (vid_capture.isOpened())
{
// Initialize frame matrix
Mat frame;
// Initialize a boolean to check if frames are there or not
bool isSuccess = vid_capture.read(frame);
// If frames are present, show it
if(isSuccess == true)
{
//display frames
imshow("Frame", frame);
}
// If frames are not there, close it
if (isSuccess == false)
{
cout << "Video camera is disconnected" << endl;
break;
}
//wait 20 ms between successive frames and break the loop if key q is pressed
int key = waitKey(20);
if (key == 'q')
{
cout << "q key is pressed by the user. Stopping the video" << endl;
break;
}
}
Once the video stream is fully processed or the user prematurely exits the loop, you release the video-capture object (vid_capture) and close the window, using the following code:
Python
# Release the objects
vid_capture.release()
cv2.destroyAllWindows()
C++
// Release video capture object
vid_capture.release();
destroyAllWindows();
Reading an Image Sequence
Processing image frames from an image sequence is very similar to processing frames from a video stream. Just specify the image files which are being read.
In the example below,
- You continue using a video-capture object
- But instead of specifying a video file, you simply specify an image sequence
- Using the notation shown below (Cars%04d.jpg), where %04d indicates a four-digit sequence-naming convention (e.g. Cars0001.jpg, Cars0002.jpg, Cars0003.jpg, etc).
- If you had specified “Race_Cars_%02d.jpg” then you would be looking for files of the form:
(Race_Cars_01.jpg, Race_Cars_02.jpg, Race_Cars_03.jpg, etc…).
All other codes described in the first example would be the same.
Python
vid_capture = cv2.VideoCapture('Resources/Image_sequence/Cars%04d.jpg')
C++
VideoCapture vid_capture("Resources/Image_sequence/Cars%04d.jpg");
Reading Video from a Webcam
Reading a video stream from a web camera is also very similar to the examples discussed above. How’s that possible? It’s all thanks to the flexibility of the video capture class in OpenCV, which has several overloaded functions for convenience that accept different input arguments. Rather than specifying a source location for a video file or an image sequence, you simply need to give a video capture device index, as shown below.
- If your system has a built-in webcam, then the device index for the camera will be ‘
0’. - If you have more than one camera connected to your system, then the device index associated with each additional camera is incremented (e.g.
1,2, etc).
Python
vid_capture = cv2.VideoCapture(0, cv2.CAP_DSHOW)
C++
VideoCapture vid_capture(0);
You might be wondering about the flag CAP_DSHOW. This is an optional argument, and is therefore not required. CAP_DSHOW is just another video-capture API preference, which is short for directshow via video input.
Writing Videos
Let’s now take a look at how to write videos. Just like video reading, we can write videos originating from any source (a video file, an image sequence, or a webcam). To write a video file:
- Retrieve the image frame height and width, using the
get()method. - Initialize a video capture object (as discussed in the previous sections), to read the video stream into memory, using any of the sources previously described.
- Create a video writer object.
- Use the video writer object to save the video stream to disk.
Continuing with our running example, let’s start by using the get() method to obtain the video frame width and height.
Python
# Obtain frame size information using get() method
frame_width = int(vid_capture.get(3))
frame_height = int(vid_capture.get(4))
frame_size = (frame_width,frame_height)
fps = 20
C++
// Obtain frame size information using get() method
Int frame_width = static_cast<int>(vid_capture.get(3));
int frame_height = static_cast<int>(vid_capture.get(4));
Size frame_size(frame_width, frame_height);
int fps = 20;
As previously discussed, the get() method from the VideoCapture() class requires:
- A single argument from an enumerated list that allows you to retrieve a variety of metadata associated with the video frames.
The metadata available is extensive, and can be found here.
- In this case, you are retrieving the frame width and height, by specifying
3(CAP_PROP_FRAME_WIDTH) and4(CAP_PROP_FRAME_HEIGHT). You will use these dimensions further below, when writing the video file to disk.
In order to write a video file, you need to first create a video-writer object from the VideoWriter() class, as shown in the code below.
Here’s the syntax for VideoWriter():
VideoWriter(filename, apiPreference, fourcc, fps, frameSize[, isColor])
The VideoWriter() class takes the following arguments:
filename: pathname for the output video fileapiPreference: API backends identifierfourcc: 4-character code of codec, used to compress the frames (fourcc)fps: Frame rate of the created video streamframe_size: Size of the video framesisColor: If not zero, the encoder will expect and encode color frames. Else it will work with grayscale frames (the flag is currently supported on Windows only).
The following code creates the video writer object, output from the VideoWriter() class. A special convenience function is used to retrieve the four-character codec, required as the second argument to the video writer object, cv2.
VideoWriter_fourcc('M', 'J', 'P', 'G')in Python.VideoWriter::fourcc('M', 'J', 'P', 'G')in C++.
The video codec specifies how the video stream is compressed. It converts uncompressed video to a compressed format or vice versa. To create AVI or MP4 formats, use the following fourcc specifications:
AVI: cv2.VideoWriter_fourcc('M','J','P','G')
MP4: cv2.VideoWriter_fourcc(*'XVID')
The next two input arguments specify the frame rate in FPS, and the frame size (width, height).
Python
# Initialize video writer object
output = cv2.VideoWriter('Resources/output_video_from_file.avi', cv2.VideoWriter_fourcc('M','J','P','G'), 20, frame_size)
C++
//Initialize video writer object
VideoWriter output("Resources/output.avi", VideoWriter::fourcc('M', 'J', 'P', 'G'),frames_per_second, frame_size);
Now that you have created a video writer object, use it to write the video file to disk, one frame at a time, as shown in the code below. Here, you are writing an AVI video file to disk, at 20 frames per second. Note how we have simplified to loop from the previous examples.
Python
while(vid_capture.isOpened()):
# vid_capture.read() methods returns a tuple, first element is a bool
# and the second is frame
ret, frame = vid_capture.read()
if ret == True:
# Write the frame to the output files
output.write(frame)
else:
print(‘Stream disconnected’)
break
C++
while (vid_capture.isOpened())
{
// Initialize frame matrix
Mat frame;
// Initialize a boolean to check if frames are there or not
bool isSuccess = vid_capture.read(frame);
// If frames are not there, close it
if (isSuccess == false)
{
cout << "Stream disconnected" << endl;
break;
}
// If frames are present
if(isSuccess == true)
{
//display frames
output.write(frame);
// display frames
imshow("Frame", frame);
// wait for 20 ms between successive frames and break
// the loop if key q is pressed
int key = waitKey(20);
if (key == ‘q’)
{
cout << "Key q key is pressed by the user.
Stopping the video" << endl;
break;
}
}
}
Finally, in the code below, release the video capture and video-writer objects.
Python
# Release the objects
vid_capture.release()
output.release()
C++
// Release the objects
vid_capture.release();
output.release();
Errors That One Might Face when Reading or Writing Videos
Video reading
While reading frames it can throw an error if the path is wrong or the file is corrupted or frame is missing.That’s why we are using an if statement inside the while loop. Which can be seen in the line, if ret == True. This way it will process it only when frames are present. Following is an example of an error log that is observed in this case. It is not the full log, only key parameters are included.
cap_gstreamer.cpp:890: error: (-2) GStreamer: unable to start pipeline in function
For wrong path:
When you provide a wrong path to a video, then it will not show any error or warning using the VideoCapture() class. The issues will arise when you will try to do any operation on the video frames. For this, you can use a simple if block for checking whether you have read a video file or not like we did in our example. That should print the following message.
Error opening the video file
Video writing
In this step various errors can occur. Most common are frame size error and api preference error. If the frame size is not similar to the video, then even though we get a video file at the output directory, it will be blank. If you are using the NumPy shape method to retrieve frame size, remember to reverse the output as OpenCV will return height x width x channels. If it is throwing an api preference error, we might need to pass the CAP_ANY flag in the VideoCapture() argument. It can be seen in the webcam example, where we are using CAP_DHOW to avoid warnings being generated.
Following are the examples of error logs:
When CAP_DSHOW is not passed:
[WARN:0]...cap_msmf.cpp(438) …. terminating async callback
When frame size is not correct:
cv2.error: OpenCV(4.5.2) :-1: error: (-5:Bad argument) in function 'VideoWriter'
> Overload resolution failed:
> - Can't parse 'frameSize'. Sequence item with index 0 has a wrong type
> - VideoWriter() missing required argument 'frameSize' (pos 5)
> - VideoWriter() missing required argument 'params' (pos 5)
> - VideoWriter() missing required argument 'frameSize' (pos 5)
Expand Your Knowledge
Summary
In this blog, you learned to read and display video streams from three different sources, using a video-capture object. Even saw how to use a video-capture object to retrieve important metadata from the video stream. We also demonstrated how to write video streams to disk, using a video-writer object. You may also find it helpful to resize video frames, or annotate them with shapes and text. For this, just modify the individual image frames.
Now that you know how to read and write videos, and are comfortable using OpenCV,keep up the pace. And keep learning.
