Calibrating a stereo camera with Python

In my last post I showed how to build a stereo camera and work with it comfortably Python as a cohesive object. Today I’ll show you how to calibrate the stereo pair so that you can rectify pictures taken with it and use it to produce maps in 3D.

The sources I’m writing about are from my StereoVision package, which contains several classes for stereo vision and 3D reconstruction and command line utilities for working with them if you don’t need to integrate them into your own program.

First I’ll show you how to use it and then I’ll explain a little bit about how you can use the code for your own purposes.

Using the program stand-alone

The code for this post is used by the utility calibrate_cameras. It can be used as a stand-alone program to calibrate photos that you’ve taken manually or can use the classes it imports to in your own programs. The output is a number of binary files containing numpy arrays that can be loaded into the StereoCalibration class at a later time. It’s used like this:

me@localhost:~> calibrate_cameras --help
usage: calibrate_cameras [-h] [--rows ROWS] [--columns COLUMNS]
                         [--square-size SQUARE_SIZE] [--show-chessboards]
                         input_folder output_folder

Read images taken with stereo pair and use them to compute camera calibration.

positional arguments:
  input_folder          Input folder assumed to contain only stereo images
                        taken with the stereo camera pair that should be
                        calibrated.
  output_folder         Folder to write calibration files to.

optional arguments:
  -h, --help            show this help message and exit
  --rows ROWS           Number of inside corners in the chessboard's rows.
  --columns COLUMNS     Number of inside corners in the chessboard's columns.
  --square-size SQUARE_SIZE
                        Size of chessboard squares in cm.
  --show-chessboards    Display detected chessboard corners.

If you’re wanting to use it following my instructions for working with a homemade stereo camera pair, you could do it like this:

me@localhost:~> capture_chessboards 0 2 50 calibration_pictures/
me@localhost:~> time calibrate_cameras --rows 9 --columns 6 --square-size 1.8 calibration_pictures/ calibration/
Reading input files...
[=======================================================================================] 100%
Calibrating cameras. This can take a while.
The average error between chessboard points and their epipolar lines is
1.10487794189 pixels. This should be as small as possible.

real    2m25.075s
user    2m25.025s
sys     0m0.070s
me@localhost:~> ls calibration/
cam_mats_left.npy      dist_coefs_right.npy  proj_mats_right.npy   trans_vec.npy
cam_mats_right.npy     e_mat.npy             rect_trans_left.npy   valid_boxes_left.npy
disp_to_depth_mat.npy  f_mat.npy             rect_trans_right.npy  valid_boxes_right.npy
dist_coefs_left.npy    proj_mats_left.npy    rot_mat.npy

Or in one step like this:

capture_chessboards --rows 9 --columns 6 --square-size 1.8 --calibration-folder calibration 0 2 50 calibration_pictures

If you run the code yourself, you’ll notice that it’s nice and quick except for the calibration itself. That requires a lot of heavy lifting, which is fortunately covered by OpenCV in C++. The whole process takes about 2.5 minutes, not including the time you need to take the pictures themselves.

Using the sources in your own program

So what if you’re wanting to work with the code on your own? Let’s take a look in the sources.

The StereoCalibrator class

The StereoCalibrator does most of the work here. It is initialized with the following signature:

calibrator = StereoCalibrator(rows, columns, square_size, image_size)

Where rows and columns are the number of inside points in the chessboard photos along the rows and columns, respectively, square_size is the size of the chessboard squares in cm and image_size is the size of the pictures in pixels.

Of course, the smartest way to use this is not to use a static image_size, but rather to read an image pair and pass the size of the images you read to the StereoCalibrator, but I leave it up to the user to decide how to work with the class.

To keep the StereoCalibrator nice and small, it doesn’t store any images. It consumes images in the form of cv2 images and stores only the detected corners. An image pair is added like this:

calibrator.add_corners((img_left, im_right))

An additional boolean argument, show_results, can be provided. If you set this, each picture will be shown overlaid with the detected chessboard corners.

After you’ve added all the image pairs you want to work with, you just calibrate the cameras based on the chessboard corners you found.

calibration = calibrator.calibrate_cameras()

This returns a StereoCalibration object that you can use for any further work that you want to do.

You can check the average error of any StereoCalibration object compared to the chessboard corners stored in a StereoCalibrator object. For example, we can check the average error of the calibration we just computed like this:

avg_error = calibrator.check_calibration(calibration)

This is measured in pixels and shows the error between the position of the unrectified chessboard points and the position the StereoCalibrator computes for them using the StereoCalibration. It should be as low as possible.

The StereoCalibration class

The StereoCalibration class is pretty simple. It can be instantiated in the following ways:

# Make an empty calibration
calibration = StereoCalibration()
# Clone a calibration from another StereoCalibration object in memory
calib_clone = StereoCalibration(calibration)
# Load stored calibration matrices from a folder
calib_loaded = StereoCalibration(input_folder=indir)

Once the StereoCalibration is populated with values you can export it to file like this:

calibration.export(outdir)

So now can calibrate your stereo camera! See the next post for using your calibrated stereo pair to produce 3D maps.

Advertisements
About

My name’s Daniel Lee. I’m an enthusiast for open source and sharing. I grew up in the United States and did my doctorate in Germany. I've founded a company for planning solar power. I've worked on analog space suit interfaces, drones and a bunch of other things in my free time. I'm also involved in standards work for meteorological data. Now I work for the German Weather Service on improving forecasts for weather and renewable power production.

Tagged with: , , , , ,
Posted in Uncategorized
117 comments on “Calibrating a stereo camera with Python
  1. Navic says:

    Thanks for these posts, very helpful! Is there an ETA on your next one showing us how to use the calibrated values?

    Like

  2. […] my last posts, I showed you how to build a stereo camera, calibrate it and tune a block matching algorithm to produce disparity maps. The code is written in Python in […]

    Like

  3. […] Get the code for the project from my repository on Github Put together your stereo camera Calibrate the cameras […]

    Like

  4. menno says:

    Hey,
    when using “calibrate_cameras”, I get:
    AttributeError: ‘module’ object has no attribute ‘computeCorrespondEpilines’
    Is this a common error?
    My goal would be later to transfer the calibration code to C++…
    Thanks! Bye, Menno

    Like

    • erget says:

      I’ve definitely never seen it. Do you have OpenCV for Python installed?

      Like

    • erget says:

      I’ve never seen it before. Do you have OpenCV for Python installed?

      Like

      • menno says:

        Yes I have that installed. Don’t know what is going wrong here. Maybe I will stick with using libfreenect for now :). I was curious if using stereovision could give me a longer depth range (kinect is only up to about 7 meters). Thanks for your post and your swift reply!

        Like

      • erget says:

        Hmm… Yes, the error message is pointing out that it can’t find that function in the module cv2, but it’s definitely in there. I would guess that something’s going wrong with cv2. As far as I’ve heard, the stand-alone scripts I provide in the package are plug-and-play: That’s how I planned them and so far I haven’t heard of anybody not being able to use them due to import errors.

        Although it is principally possible to get a longer depth range with stereo vision (you’re working with passive sensors so they observe anything they see) you will find that accuracy is inversely proportional to distance. You just don’t get enough disparity from far away. I would assume that the Kinect has a higher z-axis accuracy because it uses active remote sensing rather than the passive strategy of a stereo camera, but of course then you’re limited by the device’s power. Wish you the best of luck!

        Like

    • erget says:

      A quick follow-up note: Another reader was having similar problems and was able to fix them by installing the release version of OpenCV, rather than using its got master branch. On the branch the API is apparently different, although I can’t imagine why. So if you’d still like to try out the code you might try that – then it should run without the need for modification.

      Like

  5. Steve Zalek says:

    Hello, Daniel, and thanks for all your great work. I’m operating on an Xubuntu-14.04_64-bit system with OpenCV and your StereoVision application installed.
    I’m trying to use parts of your StereoVision application to process pairs of image files already taken with a stereovision setup (not the one you described in your other post). I have placed the input files in a common folder, and they are named “left001.bmp”, right001.mp”, “left002.bmp”, right002.bmp”, etc.
    My grid is rather large (15 x 9 interior points) and the squares are 2-inches (5.08 cm). The input folder only contains images where the grid fully shows in the pair. I run the command: calibrate_cameras –rows 15 –columns 9 –square-size 5.08 –show-chessboards CalInputImages CalOutputData
    All works well (and the found chessboard points show nicely on the screen viewer, so I know that is working well). However, I get an error at “/usr/local/lib/python2.7/dist-packages/stereovision/calibration.py”, line 233, in calibrate_cameras flags=flags)[1:]
    It indicates: TypeError: Required argument ‘distCoeffs1′ (pos 5) not found
    Are tehre some other parameters that I am mising? Would you know what this error refers to? Any help you could provide would be greatly appreciated.
    Thanks!

    Like

    • erget says:

      Hi Steve, this is definitely a strange error. It originates from OpenCV, which is complaining about a missing argument in the function call at the position reported in the stack trace. The call matches the call signature, though, which makes the whole thing very strange. I’ve definitely never had the error – it should actually only arise when using an improperly written function call.

      Since you are now the second person to report such an error, though, I’m starting to think that perhaps – however unlikely this is – OpenCV changed a few call signatures after an update. Unless you’ve changed the code in StereoVision, this would be the only cause I can think of – that OpenCV has a new version out that claims to be compatible but is in fact no longer backwards compatible. I’ll look into it and get back to you.

      Daniel

      Like

      • Steve Zalek says:

        Hi, Daniel:
        Thanks for looking into this. I forgot to mention that I was using the git-master version of openCV. Maybe there is a regression in that code (as of July 14, 2014). I may revert to version 2.4.9 and see how that works.
        Also, I only had 5 sets/pairs (left & right) of images in the folder (just trying things out) – could it be that there weren’t enough pairs of images for the software? I’m getting more pairs of calibration images prepared.
        Thanks!

        Like

      • erget says:

        Hi Steve,
        5 images are definitely not a large number – normally I use 40 or so. After a certain amount adding images doesn’t help any more but I always go for getting as much information as possible without overdoing it. Nonetheless, the function shouldn’t be producing that error because of having too few images – if it couldn’t deal with that, the error would be differently. Could you let me know what happens when you revert to the other version or OpenCV? Using an old version is of course suboptimal but it would be good to know if the problem comes from there. Thanks!

        Like

      • Steve Zalek says:

        Hi, Daniel:
        Your ‘calibrate_cameras’ code is just fine; the problem was with with my use of a development version of OpenCV. I am now using OpenCV-2.4.9 and calibration seems to work.
        I have also increased the number of calibration image pairs to 14, but I’ll have to get to a higher number than that (40+). I am using 1280×720 pixel images; could you recommend a decent value for the final calibration average error between the chessboard points and their epipolar lines? Thanks – I’m very new at this.

        Liked by 1 person

      • erget says:

        Hi Steve,
        I’m glad things are working, even if I find it confusing that OpenCV uses a different API for its release than for the development code.
        As far as collecting imagery is concerned, if you have physical access to the cameras it should be really easy – I’ve even written a tool for that purpose that I describe in this post called capture_chessboards.
        Concerning the number of calibration images you need and the acceptable error, that’s really dependent on your particular setup, the possibilities it offers, and what you consider satisfying for your application. I personally would try collecting 30-45 good calibration images and getting the error down to <10 PX, but these are by no means magical numbers. I would also be more interested in making some test point clouds for making sure that my entire chain works well before worrying about "production quality" imagery – if you go through the whole chain and get nonsense point clouds, it's probably not due to poor calibration but to something more fundamental, like swapping left and right sides or some similar, simple error. For this reason, I would first calibrate without investing too much time and effort, produce results, and then refine until I was satisfied.

        Like

  6. Steve Zalek says:

    Hi, Daniel:
    Thanks for all your info and advice. I have added more calibration images to the set (total of 44 L/R pairs) and they are all accepted without incident by the calibration process.
    The interesting thing is that my calibration error (chessboard points to epipolar lines) has now gone up to 170 pixels (with 44 pairs of images) from 113 pixels (with 14 pairs of images) – not quite the <10 pixel error that is preferable. I did try swapping the left/right lables on the images and the error doubled to ~350, so that is not the issue.
    My images are labled left(right)001.bmp, left002.bmp, etc., but some numbers are skipped (005 – 013 are skipped, and numbering resumes at 014). Could this be an issue?
    If necessary I can take some additional calibration images and substitute these in to the data set.
    Any thoughts you have would be greatly appreciated.

    Like

    • erget says:

      Having reduced accuracy is definitely a possibility when you keep on adding pictures – sometimes they just don’t improve the algorithm any more. The disjunct numbering is no problem, the function adapts to that. Have you run it using the flag to show the chessboards? It would be interesting to visually verify that they’re being successfully recognized. You do have a large pixel error, but then your images also have a high resolution 🙂

      Like

      • Steve Zalek says:

        Hi, Daniel:
        Thanks for the info. I did verify that the calibration algorithm was successfully finding the chessboard square corners – nice function! In every case the colored circles center on the chessboard square corners, except for one pair of images – I may pull these. So I have confidence that the images are being found, etc.
        Aha! I just lowered the error from 170 to 36 by removing ‘redundant’ images (images that were fairly similar to others in the cal set), and the less-than-stellar pair mentioned above. I only had to remove 4 pairs of images (out of 44) to achieve a change in error from 170 to 44, and then removng the less-than-stellar image pair dropped it further to 36.
        Knowing this I may look of a few other images to augment the cal set later, but I’m going to move on to making test point clouds, etc., to check out the entire process. Now I just need to find out how to do this… 🙂 I am reading “Programming Computer Vision wiht Python” as well.

        Like

  7. liberona91 says:

    Hi Daniel,

    My name is Paul and I am using the package you’ve provided in this tutorial, nice coding!

    When the Raspberry Pi starts making process “Calibrating cameras. This can take a while.” it take some time … you say it shouldn’t take more than 2.5 minutes to complete this process, but in my case it takes a long time (over an hour) to complete the calibration.

    Despite this detail, I am very happy because after several attempts I could finish the calibration of cameras 🙂 I had to compile OpenCV 2.4.9 version, because I threw the same error above that came out to Steve Zalek:

    “/usr/local/lib/python2.7/dist-Packages/stereovision/calibration.py”, line 233, in calibrate_cameras flags = flags) [1]
    It Indicates: TypeError: Required argument ‘distCoeffs1′ (position 5) not found

    And long before I had the error menno reported on July 11, 2014 “compute CorrespondEpilines”. This reinforces the solution you mention on July 17 … I set the cmake compilation, Release to make it, and it worked. I also took the advice of Steve Zalek and moved to version 2.4.9 … It took a while but finally it worked 🙂

    I took 50 pairs of images for calibration, visually checked with the “–show-chessboards’ argument that they are properly identified.

    Now I will continue with the tutorial of the GUI. Thank you very much for the tutorials!

    Best regards,

    Paul Liberona

    Like

  8. Andrew says:

    Hello,
    very nice coding!
    I am trying to use the package, but I receive an error though that is preventing me from using it:
    opencv error assertion failed (size.width 0 && size.height 0) in cv::imshow
    caused by:
    – stereo_camera.py line 107 in get_chessboard (self_frames(1)
    – line 80 in show_frames cv2.imshow(window, frame) (error(-215))

    I have opencv release version 2.4.9 and Python 2.7

    Did anybody already encounter this issue?

    Thanks!
    Andrew

    Like

    • Andrew says:

      Ignore my question, I didn’t realise that I was using the wrong arguments for the webcams (0 2) instead of (0 1)…
      Silly mistake by me!

      Now I have another error though: no chessboard could be found

      I am looking into this, if I solve it I will post how.

      Like

      • Andrew says:

        Ok,
        the error might have been caused by the fact that I was too far from the cameras and the algorithm was having issue to find the corners. After several tries, closer to the cameras, it worked (average error 0.92 pixels).

        Like

      • erget says:

        Glad it helped! Yeah, if the chessboards are too small they’re hard to find so the algorithm can fail. Have fun with your calibrated cameras!

        Like

  9. Jamie says:

    Hi there!

    I am attempting to use your (very well written!) code for calibrating my stereo cameras. I am struggling to get the calibrate_cameras function. Python throws a runtime error at the stereoCalibrate function:
    (line 233 on GitHub)
    “new style getargs format but argument is not a tuple”
    It’s a fairly unhelpful error message as it doesn’t refer to the argument which is causing the problem.

    Any suggestions you can think of would be much appreciated.

    Cheers.

    Jamie`

    Like

    • erget says:

      Hi Jamie,

      I’m glad you like the code and hope it’s useful for you!

      That error is from OpenCV, and I’m not exactly sure what it means. I have heard from some people that they’ve had trouble working with different versions of OpenCV – it seems like the Python API isn’t very stable yet, but I couldn’t say whether that’s true or not. What version are you working with?

      Depending on your level of familiarity with Python, it might be worthwhile to try out the calibration process step by step in a debugger or just using copy and paste in an interpreter. I know that’s more work than anybody wants to do, but it would be easier for me to understand your problem if we could see the inputs that are being passed to the function. Are you using the program in the package’s bin folder or are you building your own program on the library?

      Best,
      Daniel

      Like

  10. tasneem says:

    HELLO
    do you know what the cause of this error ,it happened during calibration

    Traceback (most recent call last):
    File “calibrate_cameras”, line 56, in
    main()
    File “calibrate_cameras”, line 52, in main
    calibrate_folder(args)
    File “C:\StereoVision-master\bin\ui_utils.py”, line 104, in calibrate_folder
    show_results=args.show_chessboards)
    File “C:\StereoVision-master\bin\calibration.py”, line 210, in add_corners
    corners = self._get_corners(image)
    File “C:\StereoVision-master\bin\calibration.py”, line 154, in _get_corners
    raise ChessboardNotFoundError(“No chessboard could be found.”)
    calibration.ChessboardNotFoundError: No chessboard could be found.

    Like

    • erget says:

      Hi there,

      The error message is honest – did you read it?

      “No chessboard could be found.”

      The algorithm’s not able to find a chessboard in the picture in question.

      Cheers,
      Daniel

      Like

  11. Morteza says:

    Hello,
    Thank you very much for your well written code, it is really interesting,
    I have a problem when running the calibration code, I get the following error, can you please help me with this:
    Traceback (most recent call last):
    File “”, line 1, in
    from stereovision.ui_utils import (find_files, ccalibrate_folder, CHESSBOARD_Arguments)

    Like

    • Morteza says:

      File “C:\Python27\lib\site-packages\stereovision\ui_utils.py”, line 44, in
      from progressbar import ProgressBar, Percentage, Bar
      ImportError: No module named progressbar

      this is the rest of error that I forgot to post, sorry

      Like

      • erget says:

        Hi, the error means that you have to install the package progressbar. It’s available on the Python Package Index, you should be able to install it with pip.

        Like

  12. Morteza says:

    Thank you very much for your quick response, it worked . now I get the following error when I reach to command find_files:

    >>> input_folder=”D:\3d scanner\Shots”
    >>> input_files=find_files(input_folder)

    Traceback (most recent call last):
    File “”, line 1, in
    input_files=find_files(input_folder)
    File “C:\Python27\lib\site-packages\stereovision\ui_utils.py”, line 68, in find_files
    files = [i for i in os.listdir(folder) if i.startswith(“left”)]
    WindowsError: [Error 123] The filename, directory name or volume label syntax is incorrect: ‘D:\x03d scanner\\Shots/*.*’

    Like

    • Morteza says:

      I changed the 3d name to sth else and it worked.

      Like

    • erget says:

      Take a close look at the error message and you’ll see that you’ve got an error in the path you entered. It looks as if the backslash you used is being interpreted as escape sequence ‘\x03’ rather than the intended ‘\3’.

      Like

  13. Morteza says:

    Hello Daniel,
    when I run capture_chessboards code I get the following error, can you please help me with this:
    OpenCV error: Assertion failed(size.width)0 && size.height)0) in cv::imshow, file C:\builds\master_PackSlaveaAddon-win32-cv12-static\opencv\modules\highgui\src\windo.cpp line 271

    Thanks

    Like

    • erget says:

      Hi, it looks like maybe you typed the error rather than copied it? In any case, OpenCV is telling you that the image does not have both height and width > 0, which it requires. Are you sure you’re loading a valid image?

      Like

      • Morteza says:

        Yes, In fact I run the code in Command prompt and there is no option to copy the errors there, I fixed the problem by switching USB ports of webcams on my laptop, these are photos of my calibration pattern:


        these are taken by Matlab, When I am using capture_chessboards code it dosent capture any photos, do you think this is because my bad calibration pattern!?

        Like

      • erget says:

        It looks like your calibration picture isn’t read, the error points to a problem with the camera itself. Check your paths or device numbers, depending on how you’re doing things.

        Like

  14. Morteza says:

    Hello Daniel,
    I am getting distCoeffs1 required argument not found error while running capture_chessboard Code, please help me with this.
    thank you very much

    Like

    • erget says:

      I can’t help very much with the information you’re giving me, sorry! Would need to know how you’re using it, etc. Have you looked at the API of the function in question?

      Like

      • Morteza says:

        I am running the code in windows command prompt, I am getting an error which says distCoeffs1 are not found, I checked the API and it showed that the distCoeffs is related to distortion coefficients but I couldn’t figure it out how to compute distortion coefficients, the picture I sent shows how I run the code in command prompt, please help me with this,
        Thank you very much,
        Morteza

        Like

      • erget says:

        Ah, sorry. I hadn’t seen the picture.

        I thought you were using the library in your own code rather than using the tools, that was obviously the wrong assumption – normally when using the tools as you are that shouldn’t happen. I have noticed that the code doesn’t work with some later version of OpenCV, but I haven’t gotten around to updating it yet and it’ll probably take a while for that to happen. I’m not sure if this is the problem here.

        * What happens when you run this? Are you taking pictures live with the camera, and do you see the different camera perspectives on your screen?
        * Have you tried inserting the arguments in the order they’re listed in the program’s help text? It looks strange to me that you have the calibration folder before the device numbers, and also that the output folder is last. It could be that the arguments aren’t being parsed correctly.

        Daniel

        Like

  15. Morteza says:

    Hi,
    Thank you very much, you were right, the problem was with the openCV,when i installed an older version of openCV it worked perfectly and I could calibrate my cameras, now I am having another problem, when I run the tune_blockmatcher code without –bm_settings option it works fine, but when I try to save the settings it gives an error that permission denied, I run the cmd as Administrator and the error is like this:

    please help me with this.
    Thanks again
    Morteza

    Like

    • erget says:

      Hi there,

      I don’t see the error. Could you provide it?

      Generally you shouldn’t be working in any directories that require administrative privileges, so make sure you’re working in your own profile. That’s a common source of errors when receiving problems with write permissions.

      Cheers,
      Daniel

      Like

      • Morteza says:

        The error is :
        File “C:\Pythn27\Lib\site-packages\stereovision\blockmatchers.py”,line 94 in save_settings
        with open(settings_file,”w”) as settings_file:
        IOError: [Errno 13] Permission denied: ‘D:\\setting’

        Like

      • erget says:

        Yes… As I said, it’s definitely a permissions problem. You need to write to a folder where your user has write privileges, otherwise it won’t work.

        Like

  16. Morteza says:

    Hello again :),
    when I try to run the images to pointcloud code, I get the following Error,

    points = camera_pair,get_point_cloud(rectified_pair)
    File “C:\Python27\….\stereo_cameras.py”, line 147, in get-point_cloud
    colors = cv2.cvtColor(pair[0], cv2.COLOR_BGR2RGB)
    cv2.error: ..\..\..\opencv\modules\imgproc\src\color.cpp:3650: error: (-215)
    scn==3 || scn == 4 in function cv::cvtColor

    can you please help me with this? I guess the problem is with the openCV again, which version of openCV was exactly used for this Coding?
    Thank you very much.
    Morteza

    Like

    • Morteza says:

      I tried to run the code line by line this time, again I receive the same error when I reach to the command: points=camera_pair.get_point_cloud(rectified_pair)
      the error is :
      Traceback (most recent call last):
      File “”, line 1, in
      points=camera_pair.get_point_cloud(rectified_pair)
      File “C:\Python27\lib\site-packages\stereovision\stereo_cameras.py”, line 147, in get_point_cloud
      colors = cv2.cvtColor(pair[0], cv2.COLOR_BGR2RGB)
      error: ..\..\..\..\opencv\modules\imgproc\src\color.cpp:3650: error: (-215) scn == 3 || scn == 4 in function cv::cvtColor

      Like

      • Morteza says:

        Thank you Daniel,
        I Could finally figure out how the the codes work and I could generate my points clouds,
        sorry for asking too much questions.
        Morteza

        Like

      • erget says:

        Hi Morteza,

        I’m sorry I needed so long to get back to you! I’ve been traveling this week for work and had planned to write back tonight but I’m glad to hear that it’s working for you. Could you describe the solution for others?

        Cheers,
        Daniel

        Liked by 1 person

      • dipperwbf says:

        hello,I have the same problem of you,can you please help me with it?What did you do to solve it?
        Thank you very much

        Like

      • dipperwbf says:

        Hello! I have the same problem as you when I follow the instructions of the author,could you please help me with it? What did you do to solve the problem? By the way, my computer operating system is ubutnu14.04~
        Thank you very much!

        Like

      • dipperwbf says:

        hello, I’m recently running this code on my computer, and I have met the same problem, could you please tell me what have you done to solve the problem?
        Thanks you very much !

        Like

    • 方文斌 says:

      Hello! I have the same problem as you when I follow the instructions of the author,could you please help me with it? What did you do to solve the problem? By the way, my computer operating system is ubutnu14.04~
      Thank you very much!

      Like

    • 方文斌 says:

      hello, I’m recently running this code on my computer, and I have met the same problem, could you please tell me what have you done to solve the problem?
      Thanks you very much !

      Like

  17. Morteza says:

    Hi,
    you are welcome Daniel, I had several mistakes running the codes,
    1. I first installed the openCV3 and I figured out some of the functions doesn’t work on it, but by installing previous version of OpenCV this problem was solved.
    2. In writing the output files I Entered only the patches, and not the files, for output files I should have used the files with their extensions like this: D:\output.ply and not like this: D:\
    3. I shouldn’t have used the operating System drive for saving the files, the permissions were not granted there.
    Thank again for your nice Coding,
    Morteza

    Like

  18. 方文斌 says:

    Hello! I have the same problem as you when I follow the instructions of the author,could you please help me with it? What did you do to solve the problem? By the way, my computer operating system is ubutnu14.04~
    Thank you very much!

    Like

  19. […] Author’s description StereoVision in his blog […]

    Like

  20. 66bharat says:

    Hello Daniel

    I am following your blog for calibrating stereo camera and producing depth map from it. I was able to get the distortion coefficients and other parameters in my calib folder by using 100 chessboard images. I used this command in terminal in Ubuntu :
    capture_chessboards –rows 6 –columns 9 –square-size 2.6 –calibration-folder “/home/luthra/Desktop/calib” 1 0 100 “/home/luthra/Desktop/images”

    Now, I don’t know how to use those coefficients, those matrices and coefficients are in .npy format. How should I apply those coefficients to produce rectified images so that I can use those images to create disparity map.

    Could you please help me out ?

    Thanks
    Bhomik

    Like

  21. tonz1992 says:

    Hi Good day,
    I hope you do remember me, we spoke before about your stereo vision python package. I have gotten around to using this module now and currently i am performing my camera calibration offline(i.e – inputting checkerboard pattern taking from 2 cameras, baselength between cameras is about 45 cm) and i am currently using the calibrate_cameras utility to perform this calibration. However i get an average error of about 101 pixels. This is so large.!! do you by any means have any ideas as to what might be the cause? i currently took about 17 shots of checkerboard pattern from various orientations with a resolution of 5MP with a Goprohero4 camera
    thanks for any suggestions and thanks again for this package

    Like

  22. tonz1992 says:

    thank you very much for the reply i was able to get down the average error to about 38 pixels by doubling by number of samples for calibration.. however when producing the disparity maps i noticed that i could not view the sliders at the bottom of the window.. hence i cannot tune the block matcher algorithm to obtain better disparity maps.. i suspect this is due to the fact that i am using a resolution of 5MP? hence the window size fills the whole image and the sliders are at the bottom of the screen and i am not able to view them? Do you think this might be the case and if so do you know how i can get around this?
    Thanks for your help so far..

    Like

    • erget says:

      I’m not sure, it sounds like you’d be able to find out by moving the window around or resizing it. Any luck when you do that?

      Like

      • tonz1992 says:

        hi thanks for the reply…. i am not able to move the window around to see the bottom of the window where the track bars are avalibale??, is it worth going into the source code to keep the window a fixed size?

        Like

      • tonz1992 says:

        can you please advice where in the BM tuner class you are sepecifying the window height and width?

        Like

      • erget says:

        I don’t, those are the default parameters.

        Like

      • tonz1992 says:

        and also i have been playing around with the logitech c270 webcam to capture calibration images as well. when running the capture_chessboard binary.. it seems to capture just 1 left and right image and returns this error…

        File “/usr/local/bin/capture_chessboards”, line 84, in
        main()
        File “/usr/local/bin/capture_chessboards”, line 72, in main
        progress.update(progress.maxval – (args.num_pictures – i))
        File “build/bdist.linux-x86_64/egg/progressbar/__init__.py”, line 276, in update
        File “build/bdist.linux-x86_64/egg/progressbar/__init__.py”, line 251, in _need_update
        AttributeError: next_update

        Like

      • erget says:

        Try rereading the post – it talks about identifying the camera IDs, which is necessary.

        Like

      • tonz1992 says:

        yeah i know that i have identified the camera ids I am well aware of that !.. I can see the video feed from both webcams…using the show_webcams utility..yet i still get that error when running the capture_chessboards binary

        Like

      • erget says:

        Oh, I remember this issue. Check through the comments, I think there was discussion about this before – either that or an issue on Github. If I remember correctly the solution was to update progress_bar, the external library used for the UI.

        Like

      • tonz1992 says:

        Hi thank you very much for the reply… i have tried updating the package from progress bar2 to progressbar2 3.6.2 but i however am still stuck with the same problem ..i have also noticed as well that when i try to run “show_webcams 0 1 –ouput_folder calibration_pictures –interval 5 “.. the script shows the webcam feed from the 2 cameras but however does not take any pictures at all?.. and also i am not able to close the webcam video feed by pressing “q”..i have to kill the python shell to close the program.. I am so stuck with this am not sure exactly what is going on?

        Like

      • erget says:

        Sorry, I’m not able to reconstruct the error on my side. Do you have any experience with Python? If so, we might be able to open it up in an interpreter and debug it a little bit.

        Like

      • tonz1992 says:

        hi yes i do have experience with python and i have used the python interpreter and it seems that for some bizzare reasons the code gets stuck in the while loop and the images=pair.get_frames() function is not called in the show_webcams script .. and also i would like to know shouldnt the webcam feed be closed by pressing ‘q’ on the keyboard.. this doesnt work ?

        Like

      • erget says:

        It’s deeper in the call stack, it gets called by some part of the code. That’s why it’d be good to step through it line for longer.

        Like

      • tonz1992 says:

        Hi i have had a look true your source code and i think i figured why i had problems with the show_webcams utility to capture chessboard patterns.. basically the whole code seems to be nested in a while TRUE loop to capture chessboard patterns hence the webcam object cannot be closed .. a better implementation might perhaps to have a “for in range(x):” where x represents the number of pictures to be taken.. this would close the python script after exiting and free the webcam object from memory?? dont you think.

        Like

      • erget says:

        I’m pretty sure there was a reason for that, but I’d have to look into the code to understand it better and I don’t really have the time to search for it right now. Why don’t you write it up on Github as a ticket? There you can point to the specifics in the code that you mean. It’s also possible to just clone the repository and send a pull request of you think your implementation is better. If I remember correctly there’s a reason to use while rather than for but I’m open to all suggestions and review.

        Like

      • tonz1992 says:

        yes of course i would have a look as doing as you have suggested .. thank you.. and also i am currently using the tune_blockmatcher to tune my disparity maps it seems i have 2 different gui interfaces depending on how the tune_blockmatcher is started? when i supply the –use_stereobm and –bm_settings the gui interface only limits the number of parameters i can tune. are you aware of this by any chance?

        Like

      • erget says:

        Not sure what you mean. But depending on what type of block matcher you use different parameters are available.

        Like

      • tonz1992 says:

        I succeeded in doing the camera calibration and got an average error about 0.8 pixels. however my disparity map results are still very poor and i have tried tuning it with the tune_block mather gui interface but yet i am still not getting nice clean disparity maps.. have yu got any advice on this?? do i need to do some sort of image-processing on the images that i intend to reconstruct in 3D first ?

        Like

      • erget says:

        I’ve explained it as extensively as I know how in the blog and video.

        Like

      • tonz1992 says:

        hi yeah i know i have read the blog and watched your video .. I was just wondering if from your experience you could suggest any ideas that could lead to poor disparity maps despite having good camera calibration.. i am also using objects with well defined contours that don’t have lots of homogeneous surfaces. i would like to know how far away where your objects from your stereo camera roughly to obtain optimal results?

        Like

      • tonz1992 says:

        hi i would like to know if yu have used the show_webcams utility to capture checkerboard patterns

        Like

      • erget says:

        No. I used the calibration tool described in the post.

        Like

  23. tonz1992 says:

    have yu by any chance experienced that error before??. i only notice that when i use show_

    Like

  24. tonz1992 says:

    I also notice when i use the show_webcams utility to view the left and right webcam.. i have to press “q” quite a couple of times to close the webcam object ?

    Like

  25. tonz1992 says:

    Hi hello i hope all is well. i have read extensively your blog and also watched the video you had on youtube.. really good material and thanks for that. I am however still having issues getting good disparity maps despite having good camera calibration results with average error of about 1 pixels. I noticed in your video even before you tuned the parameters of the semi global block matching algorithm the disparity maps in its default state already gave somewhat good results for example ( with the 3d reconstruction of the chair) you could already see the contour of the object in the disparity map even before tuning the parameters. however with my results these does not seem to be the case and tuning the parameters has not really improved my results. I am kind of stuck with this. could you please suggest any tips or shed any light on the matter?
    Thank you for your help

    Like

  26. Eric FS says:

    Hello.
    First, thank you for this awesome material.
    I am following your instructions on the homemade stereo camera pair, but after I take the chessboard pictures and calibrate (using capture_chessboards –calibrate-folder or calibrate_cameras) I get this error message:

    cv2.error: ..\..\..\..\opencv\modules\calib3d\src\calibration.cpp:1477: error: (-211) image width and height must be positive in function cvCalibrateCamera2

    Could you please help me solve this problem?

    Like

    • erget says:

      Hi Eric,
      My apologies for getting back to you so very late – I haven’t been getting alerts on comments for some reason. Not an excuse, but at least you know why 🙂
      I’m not sure what happened here, except for possibly the fact that the OpenCV API has changed. I have an open pull request on the code base on GitHub and one of the issues it’ll fix is a change in the order of arguments effected by a newer OpenCV release. Maybe that’ll fix your problem? You can find it by going to the repository’s page on GitHub.
      Cheers,
      Daniel

      Like

  27. Scott says:

    Thanks for writing this post, it’s the only comprehensive guide to stereo calibration with python that I’ve found. I know it’s old, but I was wondering if you had any advice for solving the error that a couple people encountered involving the distCoeffs missing
    ” line 233, in calibrate_cameras flags=flags)[1:]
    It indicates: TypeError: Required argument ‘distCoeffs1′ (pos 5) not found”
    I’m using 3.2-dev and don’t see anything in the arguments it’s looking for that would cause it to throw that error. As far as I can tell everything you’ve coded looks good
    >>> help(cv2.stereoRectify)
    Help on built-in function stereoRectify:
    stereoRectify(…)
    stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imageSize, R, T[, R1[, R2[, P1[, P2[, Q[, flags[, alpha[, newImageSize]]]]]]]]) -> R1, R2, P1, P2, Q, validPixROI1, validPixROI2
    In the meantime I’ll keep poking at it and if I find a solution I’ll be sure to post it

    Like

    • erget says:

      Hi Scott, I’m sorry to say that I don’t have an idea here. I also don’t currently have a stereo rig that I can fiddle around with, so it’d be hard to test for me. If you find a solution, please do post it – I’d be very interested. Thanks!

      Like

      • Scott says:

        Looking at the documentation for C++ it looks like they’ve changed cameraMatrix1, distCoeffs1, cameraMatrix2, and distCoeffs2 from output only to input/output. I was able to get it to run by setting these to numpy arrays of ones:

        (calib.cam_mats[“left”], calib.dist_coefs[“left”],
        calib.cam_mats[“right”], calib.dist_coefs[“right”],
        calib.rot_mat, calib.trans_vec, calib.e_mat,
        calib.f_mat) = cv2.stereoCalibrate(self.object_points,
        self.image_points[“left”],
        self.image_points[“right”],
        self.image_size,
        np.eye(3),
        np.ones(5),
        np.eye(3),
        np.ones(5),
        criteria=criteria,
        flags=flags)[1:]

        But now I’m running into an error where it tells me that it can’t find a checkerboard with the images it just collected.

        Like

      • erget says:

        Hmm… Alright. That might be an issue of the checkerboard itself, nothing else… I don’t think the API is very outgoing, to be honest, which is why I have all this wrapper code, but I find it unfortunate that so much has changed in it. Thanks for sharing your insights!

        Like

      • Scott says:

        Scratch what I had before. Here is the corrected working code:

        (calib.cam_mats[“left”], calib.dist_coefs[“left”],
        calib.cam_mats[“right”], calib.dist_coefs[“right”],
        calib.rot_mat, calib.trans_vec, calib.e_mat,
        calib.f_mat) = cv2.stereoCalibrate(self.object_points,
        self.image_points[“left”],
        self.image_points[“right”],
        self.image_size,
        None,
        None,
        None,
        None)[1:]

        $ calibrate_cameras –rows 6 –columns 9 –square-size 2.4 inputImages/ calibration/
        inputImages/
        Reading input files…
        [=========================================================================] 100%
        Calibrating cameras. This can take a while.
        The average error between chessboard points and their epipolar lines is
        1.82195136304 pixels. This should be as small as possible.

        Like

      • erget says:

        That’s great, thanks for sharing that! Would​you be interested in sending me a pull request contributing that code? That way you’d appear as a contributor. Otherwise I’ll roll that into the next release myself.

        Like

      • Scott says:

        It’s looking for an input, but you can feed it None for those 4 arguments and it will still output the correct matricies and coefficients.

        Like

      • Scott says:

        Go ahead and put it in the next release; I don’t have a github account and it’s not that big of a change.

        Like

  28. Scott says:

    I’ve just found the latest version on Github. Just to clarify, I wasn’t using that one and it looks like it would work fine. I was using the version you have on the pip repo that was written for 2.4.9.

    Like

  29. alian says:

    It seems not working.This is what it came out.
    C:\Users\ACER\Desktop\camera\StereoVision-master\bin>python capture_chessboards –rows 8 –columns 6 –square-size 2.65 –calibration-folder C:\Users\ACER\Desktop\camera\StereoVision-master\bin\calibration-folder 0 1 50 C:\Users\ACER\Desktop\camera\StereoVision-master\bin\output_folder
    [========================================================================] 100%
    Reading input files…
    [========================================================================] 100%
    Calibrating cameras. This can take a while.
    OpenCV Error: One of arguments’ values is out of range (image width and height must be positive) in cvCalibrateCamera2, file ..\..\..\..\opencv\modules\calib3d\src\calibration.cpp, line 1477
    Traceback (most recent call last):
    File “capture_chessboards”, line 85, in
    main()
    File “capture_chessboards”, line 81, in main
    calibrate_folder(args)
    File “build\bdist.win32\egg\stereovision\ui_utils.py”, line 110, in calibrate_folder
    File “build\bdist.win32\egg\stereovision\calibration.py”, line 241, in calibrate_cameras
    cv2.error: ..\..\..\..\opencv\modules\calib3d\src\calibration.cpp:1477: error: (-211) image width and height must be positive in function cvCalibrateCamera2

    Like

    • erget says:

      This might be fixed in the newest version. Does it work if you update, e.g. with pip install -U stereovision ?

      Like

      • alian says:

        I had download stereovision 1.0.4 and install by setup.py.But it came out out the same problems just like before.Is there anything fault when I open it?

        Like

      • Ravi says:

        I’m facing the same issue on my Raspberry Pi Model 3 B+. The installed package is already the latest.
        Could this be because when I’m calibrating with a standard chessboard (using 7×7 as the input blocks), my left-right pairs in the checkerboard aren’t getting identified properly? In some cases, while the left image has a columnwise raster scan for edges, the right image has a row-wise raster scan (as seen in the overlays). Would this create an error?
        Any suggestions would be greatly appreciated!

        Like

      • erget says:

        The chessboard blocks are recognized with subpixel resolution and OpenCV doesn’t care about the camera’s scan method, so I don’t see a problem there. Are the boards blurry? What happens in interactive recognition mode?

        Like

      • Ravi says:

        I went through the Github pull and updates list and figured out the issue – it was the changed order of input arguments (image size), which I fixed. Now it works fine. Thank you!

        Like

  30. Vicki says:

    A million thanks for posting this inafmrotion.

    Like

    • Adam says:

      Hi, I have the same problem with cvCalibrateCamera2, can someone please explain what is the procedure to fix this ? Thank you

      Like

      • Adam says:

        Someone? Please?

        Like

      • Adam says:

        The fix :
        Go to /home/your_username/miniconda2/lib/python2.7/site-packages/stereovision
        Open the calibration.py file
        Go to the line 228
        Change
        ” calib.f_mat) = cv2.stereoCalibrate(self.object_points,
        self.image_points[“left”],
        self.image_points[“right”],
        calib.cam_mats[“left”],
        calib.dist_coefs[“left”],
        calib.cam_mats[“right”],
        calib.dist_coefs[“right”],
        self.image_size,


        to
        ” calib.f_mat) = cv2.stereoCalibrate(self.object_points,
        self.image_points[“left”],
        self.image_points[“right”],
        self.image_size,
        calib.cam_mats[“left”],
        calib.dist_coefs[“left”],
        calib.cam_mats[“right”],
        calib.dist_coefs[“right”],


        CTRL+S
        Fixed

        Like

      • erget says:

        Thanks for reporting this Adam. Would you be interested in sending this in to my repository via pull request?

        Like

      • Adam says:

        Done

        Like

  31. umbnich says:

    Hi Daniel! I think to try it with my Raspberry Pi and 2 PiCamera connected with IV Port.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

From the archive