In my last post I showed how to build a stereo camera and work with it comfortably Python as a cohesive object. Today I’ll show you how to calibrate the stereo pair so that you can rectify pictures taken with it and use it to produce maps in 3D.
The sources I’m writing about are from my StereoVision package, which contains several classes for stereo vision and 3D reconstruction and command line utilities for working with them if you don’t need to integrate them into your own program.
First I’ll show you how to use it and then I’ll explain a little bit about how you can use the code for your own purposes.
Using the program stand-alone
The code for this post is used by the utility calibrate_cameras. It can be used as a stand-alone program to calibrate photos that you’ve taken manually or can use the classes it imports to in your own programs. The output is a number of binary files containing numpy arrays that can be loaded into the StereoCalibration class at a later time. It’s used like this:
me@localhost:~> calibrate_cameras --help usage: calibrate_cameras [-h] [--rows ROWS] [--columns COLUMNS] [--square-size SQUARE_SIZE] [--show-chessboards] input_folder output_folder Read images taken with stereo pair and use them to compute camera calibration. positional arguments: input_folder Input folder assumed to contain only stereo images taken with the stereo camera pair that should be calibrated. output_folder Folder to write calibration files to. optional arguments: -h, --help show this help message and exit --rows ROWS Number of inside corners in the chessboard's rows. --columns COLUMNS Number of inside corners in the chessboard's columns. --square-size SQUARE_SIZE Size of chessboard squares in cm. --show-chessboards Display detected chessboard corners.
If you’re wanting to use it following my instructions for working with a homemade stereo camera pair, you could do it like this:
me@localhost:~> capture_chessboards 0 2 50 calibration_pictures/ me@localhost:~> time calibrate_cameras --rows 9 --columns 6 --square-size 1.8 calibration_pictures/ calibration/ Reading input files... [=======================================================================================] 100% Calibrating cameras. This can take a while. The average error between chessboard points and their epipolar lines is 1.10487794189 pixels. This should be as small as possible. real 2m25.075s user 2m25.025s sys 0m0.070s me@localhost:~> ls calibration/ cam_mats_left.npy dist_coefs_right.npy proj_mats_right.npy trans_vec.npy cam_mats_right.npy e_mat.npy rect_trans_left.npy valid_boxes_left.npy disp_to_depth_mat.npy f_mat.npy rect_trans_right.npy valid_boxes_right.npy dist_coefs_left.npy proj_mats_left.npy rot_mat.npy
Or in one step like this:
capture_chessboards --rows 9 --columns 6 --square-size 1.8 --calibration-folder calibration 0 2 50 calibration_pictures
If you run the code yourself, you’ll notice that it’s nice and quick except for the calibration itself. That requires a lot of heavy lifting, which is fortunately covered by OpenCV in C++. The whole process takes about 2.5 minutes, not including the time you need to take the pictures themselves.
Using the sources in your own program
So what if you’re wanting to work with the code on your own? Let’s take a look in the sources.
The StereoCalibrator class
The StereoCalibrator does most of the work here. It is initialized with the following signature:
calibrator = StereoCalibrator(rows, columns, square_size, image_size)
Where rows and columns are the number of inside points in the chessboard photos along the rows and columns, respectively, square_size is the size of the chessboard squares in cm and image_size is the size of the pictures in pixels.
Of course, the smartest way to use this is not to use a static image_size, but rather to read an image pair and pass the size of the images you read to the StereoCalibrator, but I leave it up to the user to decide how to work with the class.
To keep the StereoCalibrator nice and small, it doesn’t store any images. It consumes images in the form of cv2 images and stores only the detected corners. An image pair is added like this:
An additional boolean argument, show_results, can be provided. If you set this, each picture will be shown overlaid with the detected chessboard corners.
After you’ve added all the image pairs you want to work with, you just calibrate the cameras based on the chessboard corners you found.
calibration = calibrator.calibrate_cameras()
This returns a StereoCalibration object that you can use for any further work that you want to do.
You can check the average error of any StereoCalibration object compared to the chessboard corners stored in a StereoCalibrator object. For example, we can check the average error of the calibration we just computed like this:
avg_error = calibrator.check_calibration(calibration)
This is measured in pixels and shows the error between the position of the unrectified chessboard points and the position the StereoCalibrator computes for them using the StereoCalibration. It should be as low as possible.
The StereoCalibration class
The StereoCalibration class is pretty simple. It can be instantiated in the following ways:
# Make an empty calibration calibration = StereoCalibration() # Clone a calibration from another StereoCalibration object in memory calib_clone = StereoCalibration(calibration) # Load stored calibration matrices from a folder calib_loaded = StereoCalibration(input_folder=indir)
Once the StereoCalibration is populated with values you can export it to file like this:
So now can calibrate your stereo camera! See the next post for using your calibrated stereo pair to produce 3D maps.