Building and calibrating a stereo camera with OpenCV (<50€)

I’ve decided to try my hand at 3D reconstruction using a stereo camera. There are a few good sets of instructions on how to get started with this. I decided to take some heavy inspiration from them and will be citing them as I go in the post. I ended up not following any of the instructions precisely because I have a few goals that are not fulfilled by any single source that I could find:

• The stereo camera should be buildable from affordable, off-the-shelf materials (under 50€)
• Both cameras should be able to capture images in the near-infrared spectrum, since I would like to be able to use it to make NDVI pictures at some point if I so desire
• The cameras should be able to be calibrated in a simple workflow that bases on simple, extensible code

In this post I will be showing the first steps in a processing chain that fulfills these requirements. If you follow these instructions, you should be able to easily build your own stereo camera and calibrate it. I’ll be splitting up the process into a few separate posts to keep it from getting too long.

Materials you’ll need

The important thing about calibrated stereo photography is that the cameras have the same relationship to each other in each picture. This means that the transformation from one perspective to the other is known for each picture in each stereo pair, leaving only the work of figuring out the relationship of the objects photographed to the cameras when you’re doing 3D reconstruction. So to start, you need to build a stereo rig that will hold both of your two cameras in the same position when you’re taking pictures.

I used two cameras of the same model. This is not absolutely necessary – the whole point of calibration is empirically finding out the parameters of each camera in question, and their relation to each other. Since this is an empirical process, there is no assumption and, really, no need for the cameras to be identical. They won’t be. But using the same resolution prevents postprocessing errors due to resampling, and having the same focal length just makes the whole process easier.

I ended up settling for the Logitech C270. It cost 18.83€ at the time I wrote this post on Amazon, so for both cameras you’re paying less than 40€. I picked this camera because it’s cheap and because it’s easy to modify it if you want to capture near-infrared for e.g. NDVI pictures (for a list of cameras that have been shown to be convertible to capture near-infrared, often with instructions, see this excellent site at Public Labs).

The Lenovo C270, one of the pair I’m using for the stereo vision project. According to Public Labs this is a camera that is easy to convert to be able to see near-infrared.

The rest can be made from stuff that you probably have lying around your house, or you can buy it for under 10€ just about anywhere. They are a flat board to attach the cameras to, some universal glue to attach the cameras to it and some duct tape just for good measure. You’ll also need a flat chessboard, which can either be a real chessboard or a printed chessboard glued to a flat surface (back of a cereal box, etc.).

Building the stereo camera

The first step is to find some surface that you can attach the cameras to. The most important point here is that it’s rigid so that the cameras don’t move. I did that by gluing the cameras to a thin wooden board.

The cameras on the actual board I’ll be using. Right here they’re held on only by glue.

This held them pretty firmly, but for good measure I taped them down on both sides with some duct tape. A more permanent solution would be to drill a hole through the cameras’ clips that goes through the board and then screw them into place, but as this is a prototype I didn’t do that. In a final version, I’ll surely be using other cameras without casings and something more sturdy and lighter than an entire board like this. The point is just to have something that creates 3D images so that I can profile it on different hardware platforms and use it for some rudimentary mapping.

The glued on cameras, duct taped for extra safety.

As you can see, the duct tape holds them in place quite well.

In a final step, print your chessboard if you need to and glue it to something. I used this picture, generiously provided on Martin Peris’ blog (which, by the way, was a large inspiration for pretty much everything in this post):

Sample chessboard to print.

After that, connect the cameras to your computer and you’re done! Toaster and fruit not included.

The entire setup – the camera rig, plus the printed chessboard taped to another board.

Capturing calibration pictures

I’ve already mentioned that I took heavy inspiration from Martin Peris. In fact, if you go to his page, you’ll find working code that will do almost all of the steps I describe in this post. I decided to write my own code nonetheless, not because the code was bad, but because the workflow is not as clear, some parts of the implementation are left to the user, and it’s in C, using the old OpenCV C API. I wrote my implementation in Python for easier understanding and maintenance, and also because I’ll probably reuse the StereoPair class that I used to access the cameras as a pair, either in further Python programs or as a prototype for a C++ class if I’m running it on a Pi or something similar. The StereoPair class makes it a bit simpler if you’re working with stereo cameras in Python, and it cleans up after itself by freeing up the cameras when an object is destroyed. Feel free to use it if you like!

First of all, test if everything works. I’ve written a package with many utilities for working with stereo cameras in general and especially for using them for 3D reconstruction called StereoVision. Get it from PyPI or GitHub to start working with it. For the moment, we’re mainly interested in using the cameras as a stereo pair, which can be done like this, once the package is installed:

me@localhost:~> show_webcams --help
usage: show_webcams [-h] [--output_folder OUTPUT_FOLDER] [--interval INTERVAL]
                    devices devices

Show video from two webcams. Press 'q' to exit.

positional arguments:
  devices               Device numbers for the cameras that should be accessed
                        in order (left, right).

optional arguments:
  -h, --help            show this help message and exit
  --output_folder OUTPUT_FOLDER
                        Folder to write output images to.
  --interval INTERVAL   Interval (s) to take pictures in.

When you run it without any options besides the device numbers, you should see something like this:

If your rig is set up right, the program should show something like this when you run it – the video from both perspectives in two separate windows.

Things to watch out for are using the right device numbers. My computer recognizes its built in webcam as device 0, but if I connect both cameras before starting up it’s 1, whereas the cameras are 0 and 2, respectively. Also, if you get messages that your USB controller is out of memory, try switching to use different ports. To be sure, I always connected my cameras at startup and left them connected the entire time, but it theoretically shouldn’t matter if you unplug and reconnect them from a live system.

For the calibration, you will need lots of pictures of the chessboard. It’s important that the chessboard is visible for both cameras simultaneously, and optimally it should be photographed from several different angles and positions. You can either write your own software to do that, or manually take a lot of pictures with both cameras and match them up later, or you can use capture_chessboards that takes care of all of that for you. If you want to both capture chessboards and calibrate the camera in one step, you can use the calibrate_cameras utility.

The script asks you for the details on the cameras, the number of inside corners in the chessboard’s rows and columns (the example picture I show above, for example, does not use an 8×8 board and that’s just fine), how many pictures you want and where to save it to. It then keeps on checking the pictures to see if it can find a chessboard in both camera captures. If it does, it stores both pictures in the output folder and waits five seconds so you can reposition. For my purposes, I set it to take 50 pictures.

me@localhost:~> capture_chessboards --help
usage: capture_chessboards [-h] [--rows ROWS] [--columns COLUMNS]
                           [--square-size SQUARE_SIZE]
                           [--calibration-folder CALIBRATION_FOLDER]
                           left right num_pictures output_folder

Take a number of pictures with a stereo camera in which a chessboard is
visible to both cameras. The program waits until a chessboard is detected in
both camera frames. The pictures are then saved to a file in the specified
output folder. After five seconds, the cameras are rescanned to find another
chessboard perspective. This continues until the specified number of pictures
has been taken.

positional arguments:
  left                  Device numbers for the left camera.
  right                 Device numbers for the right camera.
  num_pictures          Number of valid chessboard pictures that should be
                        taken.
  output_folder         Folder to save the images to.

optional arguments:
  -h, --help            show this help message and exit
  --rows ROWS           Number of inside corners in the chessboard's rows.
  --columns COLUMNS     Number of inside corners in the chessboard's columns.
  --square-size SQUARE_SIZE
                        Size of chessboard squares in cm.
  --calibration-folder CALIBRATION_FOLDER
                        Folder to save camera calibration to.

A selection of the stereo images my stereo rig collected.

Now you’ve got the raw data that you need to calibrate your cameras. Check out the next post to find out how to complete the last steps of the camera calibration and start shooting depth maps!

About erget

My name’s Daniel Lee. I've founded a company for planning solar power. I've worked on analog space suit interfaces, drones and a bunch of other things in my free time. I'm also involved in standards work for meteorological data. I worked for a while German Weather Service on improving forecasts for weather and renewable power production. I led the team for data ingest there before moving to EUMETSAT.

‹ Watching Netflix natively in Linux (OpenSUSE)

Calibrating a stereo camera with Python ›

Tagged with: OpenCV, python, stereo photography
Posted in Uncategorized