Calibrating a stereo camera with Python

In my last post I showed how to build a stereo camera and work with it comfortably Python as a cohesive object. Today I’ll show you how to calibrate the stereo pair so that you can rectify pictures taken with it and use it to produce maps in 3D.

The sources I’m writing about are from my StereoVision package, which contains several classes for stereo vision and 3D reconstruction and command line utilities for working with them if you don’t need to integrate them into your own program.

First I’ll show you how to use it and then I’ll explain a little bit about how you can use the code for your own purposes.

Using the program stand-alone

The code for this post is used by the utility calibrate_cameras. It can be used as a stand-alone program to calibrate photos that you’ve taken manually or can use the classes it imports to in your own programs. The output is a number of binary files containing numpy arrays that can be loaded into the StereoCalibration class at a later time. It’s used like this:

me@localhost:~> calibrate_cameras --help
usage: calibrate_cameras [-h] [--rows ROWS] [--columns COLUMNS]
                         [--square-size SQUARE_SIZE] [--show-chessboards]
                         input_folder output_folder

Read images taken with stereo pair and use them to compute camera calibration.

positional arguments:
  input_folder          Input folder assumed to contain only stereo images
                        taken with the stereo camera pair that should be
                        calibrated.
  output_folder         Folder to write calibration files to.

optional arguments:
  -h, --help            show this help message and exit
  --rows ROWS           Number of inside corners in the chessboard's rows.
  --columns COLUMNS     Number of inside corners in the chessboard's columns.
  --square-size SQUARE_SIZE
                        Size of chessboard squares in cm.
  --show-chessboards    Display detected chessboard corners.

If you’re wanting to use it following my instructions for working with a homemade stereo camera pair, you could do it like this:

me@localhost:~> capture_chessboards 0 2 50 calibration_pictures/
me@localhost:~> time calibrate_cameras --rows 9 --columns 6 --square-size 1.8 calibration_pictures/ calibration/
Reading input files...
[=======================================================================================] 100%
Calibrating cameras. This can take a while.
The average error between chessboard points and their epipolar lines is
1.10487794189 pixels. This should be as small as possible.

real    2m25.075s
user    2m25.025s
sys     0m0.070s
me@localhost:~> ls calibration/
cam_mats_left.npy      dist_coefs_right.npy  proj_mats_right.npy   trans_vec.npy
cam_mats_right.npy     e_mat.npy             rect_trans_left.npy   valid_boxes_left.npy
disp_to_depth_mat.npy  f_mat.npy             rect_trans_right.npy  valid_boxes_right.npy
dist_coefs_left.npy    proj_mats_left.npy    rot_mat.npy

Or in one step like this:

capture_chessboards --rows 9 --columns 6 --square-size 1.8 --calibration-folder calibration 0 2 50 calibration_pictures

If you run the code yourself, you’ll notice that it’s nice and quick except for the calibration itself. That requires a lot of heavy lifting, which is fortunately covered by OpenCV in C++. The whole process takes about 2.5 minutes, not including the time you need to take the pictures themselves.

Using the sources in your own program

So what if you’re wanting to work with the code on your own? Let’s take a look in the sources.

The StereoCalibrator class

The StereoCalibrator does most of the work here. It is initialized with the following signature:

calibrator = StereoCalibrator(rows, columns, square_size, image_size)

Where rows and columns are the number of inside points in the chessboard photos along the rows and columns, respectively, square_size is the size of the chessboard squares in cm and image_size is the size of the pictures in pixels.

Of course, the smartest way to use this is not to use a static image_size, but rather to read an image pair and pass the size of the images you read to the StereoCalibrator, but I leave it up to the user to decide how to work with the class.

To keep the StereoCalibrator nice and small, it doesn’t store any images. It consumes images in the form of cv2 images and stores only the detected corners. An image pair is added like this:

calibrator.add_corners((img_left, im_right))

An additional boolean argument, show_results, can be provided. If you set this, each picture will be shown overlaid with the detected chessboard corners.

After you’ve added all the image pairs you want to work with, you just calibrate the cameras based on the chessboard corners you found.

calibration = calibrator.calibrate_cameras()

This returns a StereoCalibration object that you can use for any further work that you want to do.

You can check the average error of any StereoCalibration object compared to the chessboard corners stored in a StereoCalibrator object. For example, we can check the average error of the calibration we just computed like this:

avg_error = calibrator.check_calibration(calibration)

This is measured in pixels and shows the error between the position of the unrectified chessboard points and the position the StereoCalibrator computes for them using the StereoCalibration. It should be as low as possible.

The StereoCalibration class

The StereoCalibration class is pretty simple. It can be instantiated in the following ways:

# Make an empty calibration
calibration = StereoCalibration()
# Clone a calibration from another StereoCalibration object in memory
calib_clone = StereoCalibration(calibration)
# Load stored calibration matrices from a folder
calib_loaded = StereoCalibration(input_folder=indir)

Once the StereoCalibration is populated with values you can export it to file like this:

calibration.export(outdir)

So now can calibrate your stereo camera! See the next post for using your calibrated stereo pair to produce 3D maps.

About erget

My name’s Daniel Lee. I've founded a company for planning solar power. I've worked on analog space suit interfaces, drones and a bunch of other things in my free time. I'm also involved in standards work for meteorological data. I worked for a while German Weather Service on improving forecasts for weather and renewable power production. I led the team for data ingest there before moving to EUMETSAT.

‹ Building and calibrating a stereo camera with OpenCV (<50€)

Building an interactive GUI with OpenCV ›

Tagged with: calibration, OpenCV, python, stereo calibration, stereo camera, stereo pair
Posted in Uncategorized