UZH-FPV
The UZH-FPV Drone Racing dataset ships camera-IMU calibration bags for two devices, recorded indoors and outdoors with the camera at 45° and facing forward. We ran MVIS, Kalibr and Basalt on all 8 bags and show them in the same table as the leaderboard. This is a one-off study, not part of the leaderboard or the regression suite.
There is no ground truth. Differences show where tools disagree, not who is right. The MVIS authors made every run, including Kalibr and Basalt.
IMU weights
Section titled “IMU weights”All three tools use IMU sigmas taken from the residuals in UZH’s own Kalibr results, not the MVIS defaults. UZH
publishes no measured IMU noise, and the sigmas in its imu.yaml are the same on all 8 bags (gyro 0.05, accel 0.1).
For each bag we take the RMS of Kalibr’s 3-axis residual norm and divide by √(3 × IMU rate). That gives gyro
3.0–8.5e-4 rad/s/√Hz and accel 1.1–5.2e-3 m/s²/√Hz, against the MVIS defaults of 1e-2 and 2e-2. Bias sigmas keep the
MVIS default ratio of 0.1. These are estimates, not measured noise.
The choice matters most on the Davis. On its forward bags, the difference between the indoor and outdoor MVIS rotation is 4.1 mrad with these weights and 38.9 mrad with the defaults. Snapdragon moves less, and its forward bags get slightly worse, from 2.2 to 4.2 mrad.
The rig
Section titled “The rig”| Device | Cameras | IMU | Bags |
|---|---|---|---|
| mDAVIS346 | 1 global-shutter, 346×260, equidistant | integrated, about 1000 Hz | 4 |
| Snapdragon Flight | 2 global-shutter, 640×480, equidistant | about 500 Hz | 4 |
Board: AprilGrid 5×4, tag_size 0.075 m, tag_spacing 0.2, family t36h11.
Data and UZH’s own Kalibr results: UZH-FPV datasets page. Cite Delmerico et al., “Are We Ready for Autonomous Drone Racing? The UZH-FPV Drone Racing Dataset”, ICRA 2019. License: CC BY-NC-SA 3.0.
Results
Section titled “Results”How to read this
- Spread shows repeatability across recordings, not accuracy. A tool can repeat well and still be biased.
448.50 ± 0.48 (10): median ± sd across the 10 recordings that tool solved. With 3 or 4 recordings the sd itself is rough.(3/15): this tool has 3 of the family's 15 recordings.fixed: Kalibr doesn't estimate this value. Its camera + IMU runs take intrinsics and baselines as input, here from the calibration UZH ships. The spread here isn't repeatability.shown: the tool uses a different model here, so the value has no difference next to it.other model: Kalibr's coefficients here belong to a different lens model.not in model,not reported,no run,no data,no RS model,not in one run: the tool has no value for this row, and the words say why.no runmeans the tool never ran with this sensor;not in one runmeans it ran with each sensor, but no single run holds both ends of the baseline.not run, in the recordings line: the tool has no run for the family.- Differences: the column header gives the subtraction, using stored medians. No tool is the reference.
◆: the difference is larger than the larger of the two sds. It marks disagreement, not an error in either tool.≠n: the two medians come from different numbers of recordings.≠ model: the tools' models differ, so no difference is shown.fit: reprojection error measures fit and isn't comparable across models.‡ td: MVIS splits the IMU into gyro (td) and accel (td + t_ga); Kalibr and Basalt have one timeline. Basalt − Kalibr is the like-for-like check.- Each family folds; the first is open. Click a family's title to open or close it.
No ground truth: ◆ marks disagreement, not who is right.
median ± sd (recordings) · (3/15) partial · fixed Kalibr input · shown / other model, no Δ · ≠ model, fit: no Δ · ◆ |Δ| above the larger sd · ≠n different recording counts · ‡ see How to read this
On camera + IMU families the MVIS column uses Kalibr's IMU model, not the MVIS default. The default fills fewer rows and is in a closed fold under each family.
Key and MVIS column
Swipe a table sideways for the other tools and differences.
median ± sd (recordings) · (3/15) partial · fixed Kalibr input · shown / other model, no Δ · ≠ model, fit: no Δ · ◆ |Δ| above the larger sd · ≠n different recording counts · ‡ see How to read this
On camera + IMU families the MVIS column uses Kalibr's IMU model, not the MVIS default. The default fills fewer rows and is in a closed fold under each family.
DAVIS · mDAVIS346 + IMU
MVIS 4 of 4 · Kalibr 4 of 4 · Basalt 3 of 4
Camera + IMU · 4 bags: indoor and outdoor, 45° and forward · Kalibr scale-misalignment vs MVIS (kalibr model)
Read first: All three tools use each bag's IMU weights from the residuals in UZH's own Kalibr results. Kalibr's intrinsics and baseline are inputs from the calibration UZH ships.
Notes
Like for like: Every tool starts from the camera chain UZH ships and uses the same IMU sigmas on the same bags. Kalibr keeps the camera values fixed; MVIS and Basalt refine them.
Note: Basalt has 3 of 4 recordings: its davis_indoor_forward solve failed (lever 120 mm, accel scale 0.84) and is left out.
Run config · MVIS · mvis cam-imu
mvis cam-imu -c configs/cam_imu_<bag>_kalibr_model.yaml
davis_indoor_45, commit f5cdce0
input:
format: ros1
path: <uzh-fpv>/davis/indoor_45_calib_davis_imu.bag
imu_topics:
- /davis/imu
cache_dir: detcache
board:
board_type: apriltag
rows: 4
cols: 5
tag_size: 0.075
tag_spacing: 0.2
tag_family: t36h11
camera_input:
path: <uzh-fpv>/indoor_45_calib_davis/indoor_45_calib_davis/camchain-..indoor_45_calib_davis_cam.yaml
estimate_readout: false
solve:
estimate_t_ga: true
sigma_g: 0.0003396953287453848
sigma_a: 0.001055920434500799
sigma_bg: 3.396953287453848e-05
sigma_ba: 0.00010559204345007991
output:
path: ./davis_indoor_45/real/kalibrmodel/cam_imu.json
estimate:
imu:
intrinsics: full
model: kalibrRuns (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- davis_indoor_45 · gyro 3.40e-04 · accel 1.06e-03
- davis_indoor_forward · gyro 3.08e-04 · accel 1.82e-03
- davis_outdoor_45 · gyro 5.74e-04 · accel 1.16e-03
- davis_outdoor_forward · gyro 4.87e-04 · accel 1.22e-03
Run config · Kalibr · kalibr_calibrate_imu_camera
kalibr_calibrate_imu_camera --bag input.bag --cams camchain.yaml --imu imu.yaml \
--imu-models scale-misalignment --target target.yaml --timeoffset-padding 0.1 --dont-show-reportimu.yaml for davis_indoor_45
rostopic: /davis/imu update_rate: 1000.0 accelerometer_noise_density: 0.001055920434500799 accelerometer_random_walk: 0.00010559204345007991 gyroscope_noise_density: 0.0003396953287453848 gyroscope_random_walk: 3.396953287453848e-05
Runs (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- davis_indoor_45 · gyro 3.40e-04 · accel 1.06e-03
- davis_indoor_forward · gyro 3.08e-04 · accel 1.82e-03
- davis_outdoor_45 · gyro 5.74e-04 · accel 1.16e-03
- davis_outdoor_forward · gyro 4.87e-04 · accel 1.22e-03
Run config · Basalt · basalt_calibrate_imu
basalt/build/release/basalt_calibrate_imu \
--dataset-path \
<uzh-fpv>/davis/indoor_45_calib_davis_imu.bag \
--dataset-type \
bag \
--aprilgrid \
basalt_seeds/aprilgrid_4x5_75mm.json \
--result-path \
./basalt/davis_indoor_45/ \
--no-gui \
--max-iter \
200 \
--stall-iter \
5 \
--ftol \
1e-06 \
--skip-images \
2 \
--cam-topics \
/davis/image_raw \
--imu-topic \
/davis/imu \
--init-knots \
poses \
--init-gyro-gap \
0.1 \
--init-min-inliers \
15 \
--imu-margin \
1.0 \
--gyro-noise-std \
0.0003396953287453848 \
--accel-noise-std \
0.001055920434500799 \
--gyro-bias-std \
3.396953287453848e-05 \
--accel-bias-std \
0.00010559204345007991 \
--spline-dt-ms \
10 \
--g-norm \
9.81 \
--no-mocap \
--opt-intrinsicsdavis_indoor_45, build 77daf1c
Runs (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- davis_indoor_45 · gyro 3.40e-04 · accel 1.06e-03
- davis_indoor_forward · gyro 3.08e-04 · accel 1.82e-03
- davis_outdoor_45 · gyro 5.74e-04 · accel 1.16e-03
- davis_outdoor_forward · gyro 4.87e-04 · accel 1.22e-03
cam0 – mDAVIS346 · 11 rows · ◆ 4
| Parameter | MVISKalibr IMU model | Kalibr | Basalt | MVIS− Kalibr | Basalt− Kalibr | Basalt− MVIS |
|---|---|---|---|---|---|---|
| fx px | 173.30 ± 1.78 (4) | fixed 173.03 ± 1.20 (4) | 171.62 ± 1.10 (3/4) | −1.68 ≠n | ||
| fy px | 173.15 ± 1.68 (4) | fixed 173.03 ± 1.22 (4) | 171.59 ± 1.13 (3/4) | −1.56 ≠n | ||
| cx px | 163.65 ± 0.17 (4) | fixed 163.32 ± 0.33 (4) | 163.41 ± 0.17 (3/4) | ◆ −0.24 ≠n | ||
| cy px | 135.39 ± 2.84 (4) | fixed 135.00 ± 3.07 (4) | 134.87 ± 3.52 (3/4) | −0.52 ≠n | ||
| dist k1 | −0.02739 ± 0.00892 (4) | fixed −0.03125 ± 0.00344 (4) | −0.02110 ± 0.00480 (3/4) | +0.00629 ≠n | ||
| dist k2 | 0.00107 ± 0.02757 (4) | fixed 0.00082 ± 0.00394 (4) | −0.01699 ± 0.00945 (3/4) | −0.01806 ≠n | ||
| dist k3 | −0.02174 ± 0.03113 (4) | fixed −0.00386 ± 0.00263 (4) | 0.02458 ± 0.01275 (3/4) | ◆ +0.04632 ≠n | ||
| dist k4 | 0.01230 ± 0.01119 (4) | fixed 0.00071 ± 0.00082 (4) | −0.01269 ± 0.00674 (3/4) | ◆ −0.02500 ≠n | ||
| time offset td to base IMU‡ ms | 5.836 ± 0.369 (4) | 5.813 ± 0.192 (4) | 5.823 ± 0.076 (3/4) | +0.024 | +0.010 ≠n | −0.014 ≠n |
| distance to base IMU ‖p‖ mm | 21.11 ± 2.13 (4) | 23.40 ± 1.66 (4) | 33.57 ± 2.13 (3/4) | ◆ −2.29 | ◆ +10.17 ≠n | ◆ +12.46 ≠n |
| reproj median px | 0.344 ± 0.044 (4) | 0.137 ± 0.026 (4) | not reported | fit |
base IMU – DAVIS IMU · 19 rows · ◆ 4
| Parameter | MVISKalibr IMU model | Kalibr | Basalt | MVIS− Kalibr | Basalt− Kalibr | Basalt− MVIS |
|---|---|---|---|---|---|---|
| gyro Dw[0,0] scale x | 0.99921 ± 0.00124 (4) | 0.99723 ± 0.00158 (4) | 0.98439 ± 0.00415 (3/4) | ◆ +0.00198 | ◆ −0.01284 ≠n | ◆ −0.01482 ≠n |
| gyro Dw[1,1] scale y | 1.00373 ± 0.00703 (4) | 1.00214 ± 0.00812 (4) | 0.99559 ± 0.01401 (3/4) | +0.00159 | −0.00655 ≠n | −0.00814 ≠n |
| gyro Dw[2,2] scale z | 1.00683 ± 0.00117 (4) | 1.00612 ± 0.00184 (4) | 1.00582 ± 0.00071 (3/4) | +0.00071 | −0.00030 ≠n | −0.00101 ≠n |
| gyro Dw[1,0] cross | −0.00061 ± 0.00138 (4) | −0.00059 ± 0.00195 (4) | −0.00158 ± 0.00070 (3/4) | −0.00003 | −0.00100 ≠n | −0.00097 ≠n |
| gyro Dw[2,0] cross | −0.00356 ± 0.00925 (4) | −0.00403 ± 0.00781 (4) | −0.00231 ± 0.00987 (3/4) | +0.00046 | +0.00172 ≠n | +0.00125 ≠n |
| gyro Dw[2,1] cross | −0.01766 ± 0.00448 (4) | −0.01729 ± 0.00457 (4) | −0.01908 ± 0.00419 (3/4) | −0.00037 | −0.00179 ≠n | −0.00142 ≠n |
| accel Da[0,0] scale x | 0.99955 ± 0.00268 (4) | 0.99835 ± 0.00330 (4) | 0.99381 ± 0.00492 (3/4) | +0.00120 | −0.00454 ≠n | ◆ −0.00573 ≠n |
| accel Da[1,1] scale y | 1.00067 ± 0.00257 (4) | 1.00000 ± 0.00176 (4) | 0.99358 ± 0.00362 (3/4) | +0.00067 | ◆ −0.00642 ≠n | ◆ −0.00709 ≠n |
| accel Da[2,2] scale z | 0.98507 ± 0.00318 (4) | 0.98025 ± 0.00111 (4) | 0.96037 ± 0.00453 (3/4) | ◆ +0.00482 | ◆ −0.01989 ≠n | ◆ −0.02471 ≠n |
| Tg[0,0] g-sensitivity mrad·s/m | −0.061 ± 0.097 (4) | −0.054 ± 0.054 (4) | not in model | ≠ model | ||
| Tg[0,1] g-sensitivity mrad·s/m | 0.013 ± 0.262 (4) | −0.248 ± 0.191 (4) | not in model | ≠ model | ||
| Tg[0,2] g-sensitivity mrad·s/m | 0.081 ± 0.121 (4) | 0.146 ± 0.175 (4) | not in model | ≠ model | ||
| Tg[1,0] g-sensitivity mrad·s/m | 0.035 ± 0.078 (4) | 0.206 ± 0.134 (4) | not in model | ≠ model | ||
| Tg[1,1] g-sensitivity mrad·s/m | −0.052 ± 0.108 (4) | 0.072 ± 0.050 (4) | not in model | ≠ model | ||
| Tg[1,2] g-sensitivity mrad·s/m | 0.173 ± 0.173 (4) | 0.158 ± 0.174 (4) | not in model | ≠ model | ||
| Tg[2,0] g-sensitivity mrad·s/m | 0.067 ± 0.126 (4) | −0.098 ± 0.075 (4) | not in model | ≠ model | ||
| Tg[2,1] g-sensitivity mrad·s/m | −0.074 ± 0.167 (4) | 0.013 ± 0.099 (4) | not in model | ≠ model | ||
| Tg[2,2] g-sensitivity mrad·s/m | −0.131 ± 0.214 (4) | 0.045 ± 0.012 (4) | not in model | ≠ model | ||
| t_ga gyro−accel ms | −0.514 ± 0.217 (4) | not in model | not in model |
MVIS default IMU model (calyx) · 11 of 30 rows, model-independent only
| Parameter | MVISKalibr IMU model | MVIS default | Kalibr | default− Kalibr | default− Kalibr IMU model |
|---|---|---|---|---|---|
| cam0 – mDAVIS346 | |||||
| fx px | 173.30 ± 1.78 (4) | 173.16 ± 1.76 (4) | fixed 173.03 ± 1.20 (4) | −0.14 | |
| fy px | 173.15 ± 1.68 (4) | 173.01 ± 1.67 (4) | fixed 173.03 ± 1.22 (4) | −0.14 | |
| cx px | 163.65 ± 0.17 (4) | 163.66 ± 0.19 (4) | fixed 163.32 ± 0.33 (4) | +0.02 | |
| cy px | 135.39 ± 2.84 (4) | 135.38 ± 2.83 (4) | fixed 135.00 ± 3.07 (4) | −0.01 | |
| dist k1 | −0.02739 ± 0.00892 (4) | −0.02695 ± 0.00896 (4) | fixed −0.03125 ± 0.00344 (4) | +0.00044 | |
| dist k2 | 0.00107 ± 0.02757 (4) | 0.00093 ± 0.02778 (4) | fixed 0.00082 ± 0.00394 (4) | −0.00014 | |
| dist k3 | −0.02174 ± 0.03113 (4) | −0.02199 ± 0.03132 (4) | fixed −0.00386 ± 0.00263 (4) | −0.00025 | |
| dist k4 | 0.01230 ± 0.01119 (4) | 0.01262 ± 0.01123 (4) | fixed 0.00071 ± 0.00082 (4) | +0.00032 | |
| time offset td to base IMU‡ ms | 5.836 ± 0.369 (4) | 5.872 ± 0.376 (4) | 5.813 ± 0.192 (4) | +0.059 | +0.035 |
| distance to base IMU ‖p‖ mm | 21.11 ± 2.13 (4) | 21.52 ± 1.95 (4) | 23.40 ± 1.66 (4) | −1.88 | +0.41 |
| reproj median px | 0.344 ± 0.044 (4) | 0.343 ± 0.043 (4) | 0.137 ± 0.026 (4) | fit | fit |
SNAP · Snapdragon stereo + IMU
MVIS 4 of 4 · Kalibr 4 of 4 · Basalt 4 of 4
Camera + IMU · 4 bags: indoor and outdoor, 45° and forward · Kalibr scale-misalignment vs MVIS (kalibr model)
Read first: All three tools use each bag's IMU weights from the residuals in UZH's own Kalibr results. Kalibr's intrinsics and baseline are inputs from the calibration UZH ships.
Notes
Like for like: Every tool starts from the camera chain UZH ships and uses the same IMU sigmas on the same bags. Kalibr keeps the camera values fixed; MVIS and Basalt refine them.
Run config · MVIS · mvis cam-imu
mvis cam-imu -c configs/cam_imu_<bag>_kalibr_model.yaml
snapdragon_indoor_45, commit f5cdce0
input:
format: ros1
path: <uzh-fpv>/snapdragon/indoor_45_calib_snapdragon_imu.bag
imu_topics:
- /snappy_imu
cache_dir: detcache
board:
board_type: apriltag
rows: 4
cols: 5
tag_size: 0.075
tag_spacing: 0.2
tag_family: t36h11
camera_input:
path: <uzh-fpv>/indoor_45_calib_snapdragon/indoor_45_calib_snapdragon/camchain-..indoor_45_calib_snapdragon_cam.yaml
estimate_readout: false
solve:
estimate_t_ga: true
sigma_g: 0.0007110208680826167
sigma_a: 0.0032732613574351305
sigma_bg: 7.110208680826167e-05
sigma_ba: 0.00032732613574351307
output:
path: ./snapdragon_indoor_45/real/kalibrmodel/cam_imu.json
estimate:
imu:
intrinsics: full
model: kalibrRuns (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- snapdragon_indoor_45 · gyro 7.11e-04 · accel 3.27e-03
- snapdragon_indoor_forward · gyro 8.49e-04 · accel 5.15e-03
- snapdragon_outdoor_45 · gyro 6.61e-04 · accel 3.62e-03
- snapdragon_outdoor_forward · gyro 3.00e-04 · accel 2.53e-03
Run config · Kalibr · kalibr_calibrate_imu_camera
kalibr_calibrate_imu_camera --bag input.bag --cams camchain.yaml --imu imu.yaml \
--imu-models scale-misalignment --target target.yaml --timeoffset-padding 0.1 --dont-show-reportimu.yaml for snapdragon_indoor_45
rostopic: /snappy_imu update_rate: 500.0 accelerometer_noise_density: 0.0032732613574351305 accelerometer_random_walk: 0.00032732613574351307 gyroscope_noise_density: 0.0007110208680826167 gyroscope_random_walk: 7.110208680826167e-05
Runs (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- snapdragon_indoor_45 · gyro 7.11e-04 · accel 3.27e-03
- snapdragon_indoor_forward · gyro 8.49e-04 · accel 5.15e-03
- snapdragon_outdoor_45 · gyro 6.61e-04 · accel 3.62e-03
- snapdragon_outdoor_forward · gyro 3.00e-04 · accel 2.53e-03
Run config · Basalt · basalt_calibrate_imu
basalt/build/release/basalt_calibrate_imu \
--dataset-path \
<uzh-fpv>/snapdragon/indoor_45_calib_snapdragon_imu.bag \
--dataset-type \
bag \
--aprilgrid \
basalt_seeds/aprilgrid_4x5_75mm.json \
--result-path \
./basalt/snapdragon_indoor_45/ \
--no-gui \
--max-iter \
200 \
--stall-iter \
5 \
--ftol \
1e-06 \
--skip-images \
2 \
--cam-topics \
/snappy_cam/stereo_l \
/snappy_cam/stereo_r \
--imu-topic \
/snappy_imu \
--init-knots \
poses \
--init-gyro-gap \
0.1 \
--init-min-inliers \
15 \
--imu-margin \
1.0 \
--gyro-noise-std \
0.0007110208680826167 \
--accel-noise-std \
0.0032732613574351305 \
--gyro-bias-std \
7.110208680826167e-05 \
--accel-bias-std \
0.00032732613574351307 \
--spline-dt-ms \
10 \
--g-norm \
9.81 \
--no-mocap \
--opt-intrinsicssnapdragon_indoor_45, build 77daf1c
Runs (4, gyro and accel noise density; bias sigmas are 0.1 of these):
- snapdragon_indoor_45 · gyro 7.11e-04 · accel 3.27e-03
- snapdragon_indoor_forward · gyro 8.49e-04 · accel 5.15e-03
- snapdragon_outdoor_45 · gyro 6.61e-04 · accel 3.62e-03
- snapdragon_outdoor_forward · gyro 3.00e-04 · accel 2.53e-03
cam0 – Snapdragon left · 11 rows · ◆ 3
| Parameter | MVISKalibr IMU model | Kalibr | Basalt | MVIS− Kalibr | Basalt− Kalibr | Basalt− MVIS |
|---|---|---|---|---|---|---|
| fx px | 277.79 ± 1.13 (4) | fixed 276.47 ± 1.62 (4) | 275.65 ± 1.05 (4) | ◆ −2.14 | ||
| fy px | 277.74 ± 1.36 (4) | fixed 276.25 ± 1.74 (4) | 275.58 ± 1.23 (4) | ◆ −2.15 | ||
| cx px | 317.90 ± 2.65 (4) | fixed 317.86 ± 2.35 (4) | 318.14 ± 2.74 (4) | +0.23 | ||
| cy px | 241.59 ± 4.55 (4) | fixed 242.03 ± 4.28 (4) | 241.52 ± 4.52 (4) | −0.07 | ||
| dist k1 | −0.01116 ± 0.00372 (4) | fixed −0.00972 ± 0.00798 (4) | −0.00763 ± 0.00416 (4) | +0.00353 | ||
| dist k2 | 0.02005 ± 0.00786 (4) | fixed 0.01273 ± 0.02538 (4) | 0.01801 ± 0.00673 (4) | −0.00204 | ||
| dist k3 | −0.01854 ± 0.00973 (4) | fixed −0.00579 ± 0.02407 (4) | −0.01352 ± 0.00592 (4) | +0.00502 | ||
| dist k4 | 0.00625 ± 0.00383 (4) | fixed 0.00047 ± 0.00731 (4) | 0.00358 ± 0.00205 (4) | −0.00267 | ||
| time offset td to base IMU‡ ms | −10.764 ± 4.687 (4) | −11.017 ± 4.624 (4) | −10.975 ± 4.661 (4) | +0.253 | +0.042 | −0.211 |
| distance to base IMU ‖p‖ mm | 25.61 ± 1.67 (4) | 24.41 ± 4.71 (4) | 27.51 ± 1.67 (4) | +1.19 | +3.10 | ◆ +1.91 |
| reproj median px | 0.203 ± 0.022 (4) | 0.483 ± 0.362 (4) | not reported | fit |
cam1 – Snapdragon right · 12 rows · ◆ 2
| Parameter | MVISKalibr IMU model | Kalibr | Basalt | MVIS− Kalibr | Basalt− Kalibr | Basalt− MVIS |
|---|---|---|---|---|---|---|
| fx px | 276.74 ± 1.55 (4) | fixed 275.62 ± 2.01 (4) | 275.07 ± 1.65 (4) | ◆ −1.66 | ||
| fy px | 276.56 ± 1.61 (4) | fixed 275.36 ± 2.10 (4) | 274.91 ± 1.65 (4) | −1.64 | ||
| cx px | 314.45 ± 0.23 (4) | fixed 314.75 ± 0.72 (4) | 314.81 ± 0.21 (4) | ◆ +0.36 | ||
| cy px | 235.65 ± 7.26 (4) | fixed 236.28 ± 7.47 (4) | 235.71 ± 7.09 (4) | +0.06 | ||
| dist k1 | −0.00933 ± 0.01584 (4) | fixed −0.00874 ± 0.01894 (4) | −0.00415 ± 0.01466 (4) | +0.00518 | ||
| dist k2 | 0.01249 ± 0.02393 (4) | fixed 0.01069 ± 0.02189 (4) | 0.00688 ± 0.01525 (4) | −0.00561 | ||
| dist k3 | −0.00760 ± 0.02973 (4) | fixed −0.00495 ± 0.02314 (4) | −0.00200 ± 0.01965 (4) | +0.00559 | ||
| dist k4 | 0.00167 ± 0.01338 (4) | fixed 0.00059 ± 0.00981 (4) | −0.00022 ± 0.00992 (4) | −0.00189 | ||
| baseline to Snapdragon left mm | 79.436 ± 0.486 (4) | fixed 79.501 ± 0.319 (4) | 79.555 ± 0.762 (4) | +0.119 | ||
| time offset td to base IMU‡ ms | −10.780 ± 4.699 (4) | −11.052 ± 4.638 (4) | −10.975 ± 4.661 (4) | +0.272 | +0.077 | −0.195 |
| distance to base IMU ‖p‖ mm | 57.96 ± 1.57 (4) | 58.76 ± 2.97 (4) | 59.47 ± 2.38 (4) | −0.80 | +0.71 | +1.51 |
| reproj median px | 0.201 ± 0.043 (4) | 0.405 ± 0.497 (4) | not reported | fit |
base IMU – Snapdragon IMU · 19 rows · ◆ 7
| Parameter | MVISKalibr IMU model | Kalibr | Basalt | MVIS− Kalibr | Basalt− Kalibr | Basalt− MVIS |
|---|---|---|---|---|---|---|
| gyro Dw[0,0] scale x | 1.00405 ± 0.00194 (4) | 1.00395 ± 0.00431 (4) | 1.00351 ± 0.00102 (4) | +0.00010 | −0.00044 | −0.00054 |
| gyro Dw[1,1] scale y | 0.99791 ± 0.00213 (4) | 0.99951 ± 0.00671 (4) | 0.99392 ± 0.00339 (4) | −0.00161 | −0.00559 | ◆ −0.00399 |
| gyro Dw[2,2] scale z | 0.99792 ± 0.00185 (4) | 0.99811 ± 0.00398 (4) | 0.99402 ± 0.00409 (4) | −0.00019 | ◆ −0.00409 | −0.00390 |
| gyro Dw[1,0] cross | 0.00070 ± 0.00030 (4) | −0.00034 ± 0.00046 (4) | 0.00018 ± 0.00151 (4) | ◆ +0.00104 | +0.00052 | −0.00052 |
| gyro Dw[2,0] cross | 0.00606 ± 0.00085 (4) | 0.00704 ± 0.00538 (4) | 0.00413 ± 0.00131 (4) | −0.00098 | −0.00290 | ◆ −0.00192 |
| gyro Dw[2,1] cross | 0.00014 ± 0.00075 (4) | −0.00028 ± 0.00103 (4) | −0.00002 ± 0.00127 (4) | +0.00042 | +0.00026 | −0.00016 |
| accel Da[0,0] scale x | 0.99841 ± 0.00172 (4) | 1.00034 ± 0.00383 (4) | 0.98999 ± 0.00517 (4) | −0.00193 | ◆ −0.01035 | ◆ −0.00842 |
| accel Da[1,1] scale y | 0.99930 ± 0.00134 (4) | 0.99910 ± 0.00291 (4) | 0.99296 ± 0.00411 (4) | +0.00021 | ◆ −0.00613 | ◆ −0.00634 |
| accel Da[2,2] scale z | 0.98925 ± 0.00221 (4) | 0.99085 ± 0.00575 (4) | 0.97820 ± 0.00363 (4) | −0.00160 | ◆ −0.01265 | ◆ −0.01105 |
| Tg[0,0] g-sensitivity mrad·s/m | 0.008 ± 0.082 (4) | 0.045 ± 0.201 (4) | not in model | ≠ model | ||
| Tg[0,1] g-sensitivity mrad·s/m | −0.089 ± 0.066 (4) | 0.267 ± 0.073 (4) | not in model | ≠ model | ||
| Tg[0,2] g-sensitivity mrad·s/m | 0.046 ± 0.146 (4) | 0.165 ± 0.104 (4) | not in model | ≠ model | ||
| Tg[1,0] g-sensitivity mrad·s/m | 0.163 ± 0.138 (4) | 0.034 ± 0.234 (4) | not in model | ≠ model | ||
| Tg[1,1] g-sensitivity mrad·s/m | 0.044 ± 0.107 (4) | 0.038 ± 0.183 (4) | not in model | ≠ model | ||
| Tg[1,2] g-sensitivity mrad·s/m | −0.074 ± 0.401 (4) | −0.341 ± 0.443 (4) | not in model | ≠ model | ||
| Tg[2,0] g-sensitivity mrad·s/m | 0.029 ± 0.128 (4) | −0.053 ± 0.125 (4) | not in model | ≠ model | ||
| Tg[2,1] g-sensitivity mrad·s/m | 0.022 ± 0.094 (4) | 0.022 ± 0.219 (4) | not in model | ≠ model | ||
| Tg[2,2] g-sensitivity mrad·s/m | 0.057 ± 0.207 (4) | −0.159 ± 0.229 (4) | not in model | ≠ model | ||
| t_ga gyro−accel ms | −0.832 ± 0.124 (4) | not in model | not in model |
MVIS default IMU model (calyx) · 23 of 42 rows, model-independent only
| Parameter | MVISKalibr IMU model | MVIS default | Kalibr | default− Kalibr | default− Kalibr IMU model |
|---|---|---|---|---|---|
| cam0 – Snapdragon left | |||||
| fx px | 277.79 ± 1.13 (4) | 277.77 ± 1.11 (4) | fixed 276.47 ± 1.62 (4) | −0.01 | |
| fy px | 277.74 ± 1.36 (4) | 277.72 ± 1.33 (4) | fixed 276.25 ± 1.74 (4) | −0.02 | |
| cx px | 317.90 ± 2.65 (4) | 317.89 ± 2.64 (4) | fixed 317.86 ± 2.35 (4) | −0.01 | |
| cy px | 241.59 ± 4.55 (4) | 241.59 ± 4.56 (4) | fixed 242.03 ± 4.28 (4) | 0.00 | |
| dist k1 | −0.01116 ± 0.00372 (4) | −0.01117 ± 0.00369 (4) | fixed −0.00972 ± 0.00798 (4) | −0.00001 | |
| dist k2 | 0.02005 ± 0.00786 (4) | 0.02002 ± 0.00785 (4) | fixed 0.01273 ± 0.02538 (4) | −0.00003 | |
| dist k3 | −0.01854 ± 0.00973 (4) | −0.01849 ± 0.00969 (4) | fixed −0.00579 ± 0.02407 (4) | +0.00005 | |
| dist k4 | 0.00625 ± 0.00383 (4) | 0.00625 ± 0.00381 (4) | fixed 0.00047 ± 0.00731 (4) | −0.00001 | |
| time offset td to base IMU‡ ms | −10.764 ± 4.687 (4) | −10.794 ± 4.704 (4) | −11.017 ± 4.624 (4) | +0.223 | −0.030 |
| distance to base IMU ‖p‖ mm | 25.61 ± 1.67 (4) | 25.62 ± 1.70 (4) | 24.41 ± 4.71 (4) | +1.20 | +0.01 |
| reproj median px | 0.203 ± 0.022 (4) | 0.203 ± 0.022 (4) | 0.483 ± 0.362 (4) | fit | fit |
| cam1 – Snapdragon right | |||||
| fx px | 276.74 ± 1.55 (4) | 276.73 ± 1.52 (4) | fixed 275.62 ± 2.01 (4) | −0.01 | |
| fy px | 276.56 ± 1.61 (4) | 276.54 ± 1.59 (4) | fixed 275.36 ± 2.10 (4) | −0.02 | |
| cx px | 314.45 ± 0.23 (4) | 314.44 ± 0.25 (4) | fixed 314.75 ± 0.72 (4) | 0.00 | |
| cy px | 235.65 ± 7.26 (4) | 235.65 ± 7.25 (4) | fixed 236.28 ± 7.47 (4) | 0.00 | |
| dist k1 | −0.00933 ± 0.01584 (4) | −0.00931 ± 0.01585 (4) | fixed −0.00874 ± 0.01894 (4) | +0.00002 | |
| dist k2 | 0.01249 ± 0.02393 (4) | 0.01253 ± 0.02397 (4) | fixed 0.01069 ± 0.02189 (4) | +0.00004 | |
| dist k3 | −0.00760 ± 0.02973 (4) | −0.00761 ± 0.02978 (4) | fixed −0.00495 ± 0.02314 (4) | −0.00002 | |
| dist k4 | 0.00167 ± 0.01338 (4) | 0.00167 ± 0.01340 (4) | fixed 0.00059 ± 0.00981 (4) | 0.00000 | |
| baseline to Snapdragon left mm | 79.436 ± 0.486 (4) | 79.435 ± 0.484 (4) | fixed 79.501 ± 0.319 (4) | −0.002 | |
| time offset td to base IMU‡ ms | −10.780 ± 4.699 (4) | −10.810 ± 4.716 (4) | −11.052 ± 4.638 (4) | +0.242 | −0.030 |
| distance to base IMU ‖p‖ mm | 57.96 ± 1.57 (4) | 58.00 ± 1.55 (4) | 58.76 ± 2.97 (4) | −0.77 | +0.03 |
| reproj median px | 0.201 ± 0.043 (4) | 0.202 ± 0.044 (4) | 0.405 ± 0.497 (4) | fit | fit |
Reading these results
Section titled “Reading these results”- One Basalt solve is left out. On
davis_indoor_forwardwith these weights, Basalt gives a 120 mm lever and an accel scale of 0.84, so its Davis column has 3 of 4 bags. At Basalt’s default noise that bag is plausible, butsnapdragon_indoor_45then fails with a broken spline. - Basalt’s Davis distance stands out. Its camera-to-IMU distance is 33.6 mm, against 23.2 mm from Kalibr and 22.8 mm from MVIS on the same 3 bags. We haven’t established why. At default noise Basalt’s lever z is 14–25 mm.
- Each device’s 4 bags form one family, because the camera and IMU are on the same device. That assumes the camera-IMU geometry stayed the same across recordings, which we can’t confirm.
- Snapdragon’s time offset really does differ between bags, with an sd near 4.7 ms in every tool, so a small difference between tools on that row says little about each bag.
- Rotations aren’t in this table, the same as on the leaderboard, because the IMU models define the IMU frame differently.
Results files
Section titled “Results files”The calibration numbers and configs for all three tools, including the Basalt default-noise run, are on the Hub at
calibench-vi/datasets.
The repository is gated, and access is granted once you accept its terms.