Help building camera config

[jsreveal]

Hi guys,

I
I have a setup with 2 cameras, where I got the following calibration results using kalibr:

cam0:
  cam_overlaps: [1]
  camera_model: pinhole
  distortion_coeffs: [0.1506471776407054, 0.02697762642670598, -0.15075232293777416, 0.09723065619467497]
  distortion_model: equidistant
  intrinsics: [853.9786047047263, 850.6635666587451, 623.6791993312835, 327.91240527677644]
  resolution: [1280, 720]
  rostopic: /cam0/image_raw
cam1:
  T_cn_cnm1:
  - [0.9999875867211632, 0.0030287917965356446, 0.003956364977832583, 0.002548487103892417]
  - [-0.001816006709730414, 0.9609653000974285, -0.27666296125121226, 0.12680778366673687]
  - [-0.004639883965660773, 0.27665234217138446, 0.9609588716734306, 0.05905749861794551]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [0]
  camera_model: pinhole
  distortion_coeffs: [0.14244013737908465, 0.08932156242606772, -0.18735030577650047, -0.07640393402056334]
  distortion_model: equidistant
  intrinsics: [858.9379381563921, 857.9349666301277, 624.5076685707864, 410.189775368046]
  resolution: [1280, 720]
  rostopic: /cam1/image_raw

Can you please confirm if the correct AirSLAM camera configuration based on the previous kalibr calibration results would be the following:

Calibration
distortion_type: 2  # 0 for undistorted inputs, 1 for radial-tangential: [k1, k2, p1, p2, k3], 2 for equidistant/fisheye:  [k1, k2, k3, k4, 0].
cam0:
  intrinsics: [853.978, 850.663, 623.679, 327.912] # fx, fy, cx, cy
  distortion_coeffs: [0.150647177, 0.0269776264, -0.15075232, 0.097230656,0]
  T_type: 1           # 0 for Euroc format, the following T is Tbc. 1 for Kalibr format, the following T is Tcb
  T: 
  - [1.0, 0.0, 0.0, 0.0]
  - [0.0, 1.0, 0.0, 0.0]
  - [0.0, 0.0, 1.0, 0.0]
  - [0.0, 0.0, 0.0, 1.0]
cam1:
  intrinsics: [858.937, 857.934, 624.507, 410.189] 
  distortion_coeffs: [0.14244013, 0.08932156, -0.18735030, -0.07640393,0]
  T_type: 1           
  T: 
  - [0.9999875867211632, 0.0030287917965356446, 0.003956364977832583, 0.002548487103892417]
  - [0.999598781151, 0.0130119051815, 0.0251588363115, 0.0453689425024]
  - [-0.004639883965660773, 0.27665234217138446, 0.9609588716734306, 0.05905749861794551]
  - [0.0, 0.0, 0.0, 1.0]

I am getting the following message for all video frames of the video that I want to map using that configuration:

Not enough stereo points to initialize!
good stereo point = 0

Thank you

[xukuanHIT]

@jsreveal Hi, what type of stereo camera you are using? Typically, the rotation between the stereo cameras should be close to the identity matrix, with translation in the x-direction equal to the baseline distance between the two cameras, and near zero in the y and z directions. However, your parameters seem unusual—this could mean either the camera setup is not a conventional stereo camera, or the calibration was not done correctly.

[jsreveal]

understand. I am trying to build a setup with two cameras, manually.

had a camera above the other with the following setup:

got better results with the following setup, but still had to decrease min_init_stereo_feature to 20, and mapping is not working that much well:

• [0.9758825287742534, -0.01108600240491, 0.21801465680968526, -0.16405935922242404]
• [0.019970001767385614, 0.9990556202129569, -0.038588427673497364, -0.014351648305129722]
• [-0.2173809767725252, 0.042011525461240415, 0.9751823125271707, 0.036471440297261]
• [0.0, 0.0, 0.0, 1.0]

Should the be fully parallel in y? I find it much more difficult to calibrate them in kalibr that way.

[xukuanHIT]

The two cameras need to be aligned along the x-axis because horizontal parallax is needed to triangulate feature points. It seems there might be an issue with the calibration; you might want to first verify the accuracy of the stereo calibration results.

[jsreveal]

Ok, so I improved camera setup and calibration. there was some desynchronization in the frames timestamp from the two cameras.

This is what I got now:

cam0 reprojection error: [0.000006, 0.000031] +- [0.724156, 1.371513]
cam1 reprojection error: [-0.000040, 0.000019] +- [0.727362, 1.295480]

cam0:
  cam_overlaps: [1]
  camera_model: pinhole
  distortion_coeffs: [0.21349864755544656, -0.06006471895104325, 0.2166042859853919, -0.15794506984534448]
  distortion_model: equidistant
  intrinsics: [687.5979535034684, 690.7357049306729, 669.4072966657725, 423.3612554324596]
  resolution: [1280, 720]
  rostopic: /cam0/image_raw

cam1:
  T_cn_cnm1:
  - [0.9995402800020964, -0.0044984549454216115, 0.029983204572364485, 0.14969594088499777]
  - [0.00509030633802063, 0.9997931272719064, -0.019692420908569274, 0.00297247191981763]
  - [-0.029888416396814232, 0.01983599160513941, 0.9993564008912561, 0.006399137060623754]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [0]
  camera_model: pinhole
  distortion_coeffs: [0.25661803310881176, -0.1782796043279173, 0.37808681435669683, -0.2645000388737464]
  distortion_model: equidistant
  intrinsics: [708.9023839428361, 708.0366016992732, 637.2357227505979, 426.778900384115]
  resolution: [1280, 720]
  rostopic: /cam1/image_raw

however it still does not work. I keep not being able to detect stereo points.

Here a link with the video I am using, and respective synchronized frames: https://drive.google.com/file/d/1VjcUgzMTz367qGr_iqWj8Z5kEjVOqC80/view?usp=drive_link

thank you,
João

[xukuanHIT]

I used the provided parameters and images for stereo rectification with OpenCV’s stereo rectification function, but the results were poor (check the corresponding pixels on the chess board), indicating that the calibration parameters may not be reliable. Theoretically, after stereo rectification, corresponding pixels should be aligned on the same horizontal line.

[jsalada-dr]

Thanks for that!

From your knowledge, could this be a limitation from kalibr? Can you recommend any stereo camera that you know to have been correctly calibrated? Thank you

[xukuanHIT]

I'm not sure if this is an issue with Kalibr or the camera itself. We conducted live demo experiments using the RealSense (D435, D455) and found the results to be quite satisfactory.

[jsreveal]

Ok, so I have done improvements on the setup. Cameras were still desynchronized so got them synchronized, and also created a calibration target with the original size (A0). this got me to very good reprojection errors, compared to what I had:

cam0:
reprojection error: [-0.000014, 0.000003] +- [0.623240, 0.736569]

cam1:
reprojection error: [0.000014, -0.000003] +- [0.641612, 0.729096]

With this I am able to finally get AirSlam to clearly detect stereo points, using Euroc AirSlam Config. However I am not able to close the loop of the map I am trying to create, where I transverse a room two times using always the same path. It feels clearly that is does not have any sense of any transversed points in the past during the mapping, that could help to correct any error, and also the that the curvature angles are not being correctly computed:

Is there any configuration change that could improve this outcome? What do you suggest?

Attaching the following data:

Kalibr calibration file: calibration.txt
AirSlam config: vo_airslamconfig.txt
Stereo camera room transversal frames: https://drive.google.com/file/d/1BQq_PCBPcz_nttO8GxrWrGHbYtljWMBi/view?usp=sharing

[jsreveal]

@xukuanHIT would really appreciate some insights, if possible, please?

thanks a lot

[xukuanHIT]

Loop closure detection is part of the offline processing step, which you can refer to ”Map Optimization“ in the README. Regarding cumulative error, I suspect it might still be an issue with camera calibration, but I currently don't have time to verify this. You can follow the method I did above to visualize the results of stereo rectification.

[jsreveal]

thanks for the insights! do you believe that by adding imu sensors data, error could be reduced?

[jsreveal]

Loop closure detection is part of the offline processing step, which you can refer to ”Map Optimization“ in the README. Regarding cumulative error, I suspect it might still be an issue with camera calibration, but I currently don't have time to verify this. You can follow the method I did above to visualize the results of stereo rectification.

also would you be able to kindly sharethe code that you used to verify kalibr calibration into opencv calibration verification?

[xukuanHIT]

Hi, this is the code.
fisheye_stereo.zip

[jsreveal]

Totally managed to put this working. Thanks for all the support.