Camera–IMU calibration
Use this page to solve cameras and IMUs in one graph. You need the recording
from step 2 and the camera YAML from
step 3. You get cam_imu.json and a calibration
YAML.
The running example is recording 01 of our VI-Rig with 4 IMUs and 4 cameras (dataset page):
- Cameras: a T265 fisheye pair (global shutter) and an ELP stereo pair (rolling shutter).
- IMUs: Xsens MTi-100, MicroStrain GX3-25 and GX3-35, and the T265 IMU.
The solve estimates, all at once:
- Extrinsics: where each camera and IMU sits relative to the base IMU (lever arm and rotation).
- Time offset
td: the clock delay of each camera and each extra IMU to the base IMU. - IMU intrinsics: scale and misalignment of each gyroscope (
Dw) and accelerometer (Da). t_ga: the delay between an IMU’s own gyroscope and accelerometer samples.- Camera intrinsics: refined from the step 3 result, not held fixed.
- Biases of every gyroscope and accelerometer.
1. Point at the camera prior
Section titled “1. Point at the camera prior”Step 3 (Camera-only calibration) writes a camera YAML. Name it in the cam-IMU config:
camera_input: path: camera_prior.yaml- What it gives: the starting intrinsics of each camera (the prior).
- One file, in
cam_topicsorder: join the T265 and ELP results of step 3. MVIS sorts the cameras byidand matches them tocam_topicsby order. So here the T265 fisheyes areid0 and 1, the ELP left and right 2 and 3. - Our example’s prior: camera_prior.yaml.
It joins earlier camera-only results and uses
equidistantfor the T265, which gives the same result askb4on this lens (step 3). So the report lists the T265 cameras asequidistant. - What happens next: the cam-IMU solve refines the intrinsics (
estimate.camera.intrinsics: true). It solves the extrinsics and time offsets itself; the prior’s extrinsics are not used.
2. Write the config
Section titled “2. Write the config”This is the config of the VI-Rig 4 IMUs + 4 cameras example, recording 01. The highlighted lines matter for this step.
input: format: ros1 path: 4imus_4cams_01.bag cam_topics: - /t265/fisheye1/image_raw - /t265/fisheye2/image_raw - /elp/split_sync_image_node/left/image_raw - /elp/split_sync_image_node/right/image_raw imu_topics: - /imu/data - /gx3_25/data - /gx3_35/imu/data - /t265/imu window_mode: motion segment_length: 80.0board: board_type: apriltag rows: 6 cols: 6 tag_size: 0.088 tag_spacing: 0.3 tag_family: t36h11 heights_file: board_shape_4imus_4cams.yamlcamera_input: path: camera_prior.yaml cameras: - {} - factor_hz: 4.0 - estimate_readout: true factor_hz: 4.0 - estimate_readout: true factor_hz: 4.0solve: max_iterations: 40 corner_subsample: two_anti estimate_t_ga: true sigma_t_ga: 0.01 vision_pose: imu sigma_g: 0.00065 sigma_a: 0.0026 sigma_bg: 0.001 sigma_ba: 0.002estimate: board_points: true camera: intrinsics: true tr: false td: true extrinsics: true imu: intrinsics: dw_da model: calyx extrinsics: true td: trueoutput: path: cam_imu.jsonWhat the highlighted lines do:
input: a ROS1 bag. The firstimu_topicsentry (/imu/data, the Xsens) is the base IMU. Every other sensor is solved relative to it.window_mode: motion: MVIS uses thesegment_lengthseconds (here 80 s) with the most IMU motion.board.heights_file+estimate.board_points: true: solve the board shape too, starting from a measured shape.board_shape_4imus_4cams.yamlships with MVIS inexamples/regression/mvis_elp/. Our board is not flat; a flat solve shifts the gyro scale and the lever arm (why).solve.corner_subsample: two_anti: use two opposite corners of each tag (top-left and bottom-right). Our config sets it for speed. The cam-IMU default isall(every corner).solve.estimate_t_ga: true,sigma_t_ga: 0.01: solvet_gafor each IMU. Keep this on.solve.vision_pose: imu: moves the pose to each corner’s capture time with the raw base-IMU samples. It is the default.sigma_g,sigma_a,sigma_bg,sigma_ba: gyro and accel noise, and their bias random walk.estimate.camera: intrinsics free,tdand extrinsics solved.estimate.imu: IMU intrinsicsdw_dawith thecalyxmodel (see section 3), plus extrinsics andtdof the extra IMUs.
The other lines belong to later steps:
camera_input.cameras: one entry per camera, incam_topicsorder. The three cameras after the first enter the graph at 4 frames per second (factor_hz), which keeps the solve small. The first camera keeps the cam-IMU default of 10 Hz.- ELP readout lines (
estimate_readout,tr):estimate.camera.tr: falseis the rig-wide default and holds the readout fixed. The per-cameraestimate_readout: trueon the two ELP entries overrides it, so the run solves the ELP readout and keeps the T265 (global shutter) at zero. Details in step 5, Rolling-shutter calibration. - The three extra IMUs: step 6, Multi-IMU spatial-temporal calibration.
For your own rig, change the topics, the board and the prior path. Keep the rest as a starting point.
3. Turn on IMU intrinsics
Section titled “3. Turn on IMU intrinsics”estimate.imu.intrinsics sets how much of each IMU model we solve:
| Value | What is solved |
|---|---|
pinned | nothing: Dw and Da stay at their start values |
dw_da | scale and misalignment of gyro (Dw) and accel (Da). Our setting, and the default |
dw_da_r | as dw_da, plus the gyro–accel rotation. Same as dw_da for calyx, which keeps that rotation inside Dw |
full | as dw_da_r, plus Tg (how linear acceleration leaks into the gyro) |
- A bool also works:
truemeansfull,falsemeanspinned. estimate.imu.modelpicks the model family:calyx(default),kalibrorrpng.- Priors: freed terms get a weak prior.
solve.sigma_dw_dadefaults to 0.1,solve.sigma_tgto 0.05.
estimate: imu: intrinsics: dw_da # pinned | dw_da | dw_da_r | full model: calyx # calyx | kalibr | rpngWhat the terms mean: IMU model. Why it pays off: Why calibrate the IMU.
4. Run
Section titled “4. Run”mvis cam-imu -c virig_4imus_4cams_cam_imu.yaml- Other bag:
--bag <file>replaces the config’s source with that ROS1 bag. - Time: 22 min in our run (MVIS 0.1.0, 2026-11-12), on 16 CPU cores with
one thread for the linear-algebra libraries, on a machine under heavy load.
Expect less on your machine. About 12 min of that is AprilTag detection on
the four cameras; the solve stages take about 11 min, of which the 6
iterations take about 7 min (
solve_sin the report’s Timing section). - Output: in the folder of
output.path:cam_imu.json: a summary of the solve.cam_imu_calib.yaml: the full calibration, the file to deploy.cam_imu_camchain-imucam.yaml: the same rig in Kalibr’s camchain format.cam_imu_board_shape.yaml: the solved board shape (whenestimate.board_pointsis on).cam_imu_report.html: the report.cam_imu_views.json: the data of the report’s 3D views.
5. Read the result
Section titled “5. Read the result”Recording 01 gives the values below. The full table and a 3D view of the rig are on the example page.
The report. Every run writes cam_imu_report.html next to cam_imu.json.
It has these sections:
- Summary: status, reprojection median and P95, χ²/dof of the camera and
IMU terms with a grade each, and the base IMU’s
t_ga. χ²/dof near 1 means the residuals match the noise you set; well below 1 (0.13 here) means the noise settings are conservative. - 3D Geometry: the solved rig path around the board as a player, and the rig to scale.
- Cameras: per camera, intrinsics, distortion, lever arm and rotation
(log(R), rad) with their σ,
td, readout time, and the corner accounting (corners used, past the lens fold, dropped as outliers). - Board Shape: the solved corner heights as a table and a map, with the
median σ of a height. Shown because
estimate.board_pointsis on. - Reprojection Analysis: error by frame, by tag and over the board, per camera.
- Motion & Excitation: the angular velocity of the solved path, which shows
how well the motion constrains the lever arms and
td. - Base IMU, Aux IMUs, IMU Residuals: biases and their drift,
Dw,Da,Tgandt_gaof each IMU; each extra IMU’s pose andtdrelative to the base IMU. The block is titled “IMU intrinsics (full)” whatever level you solve; withdw_da, as here,Tgstays at zero. - Timing, Result files, Configuration: where the time went, the files the run wrote, and the config it read.
This is the report of the recording 01 run, unchanged except that local folder names are cut down to file names:
Open the report full screen (HTML, 12.8 MB).
Fit and geometry:
| Value | Recording 01 |
|---|---|
| Reprojection median, all cameras | 0.25 px (rms 0.52, p95 0.68) |
| cam0–cam1 baseline (T265 pair) | 63.7 mm |
cam0 td | −5.41 ms |
| cam0 lever arm | 130.8 mm |
| Board shape | 9.0 mm peak to peak, 2.2 mm RMS (72 of 144 corners seen) |
- cam0: T265 fisheye 1. cam1 is T265 fisheye 2, cam2 and cam3 are ELP left and right.
- 72 of 144 corners: with
two_anti, only two corners of each tag enter the solve. The other heights stay at the measured shape fromheights_file. - Reprojection shows how well the model fits the corners. It does not show how accurate the model is.
Base IMU (Xsens) intrinsics, diagonal terms:
| x | y | z | |
|---|---|---|---|
Dw (gyro) | 0.99813 | 0.99768 | 1.00090 |
Da (accel) | 0.99994 | 0.99642 | 1.00124 |
t_ga per IMU:
| IMU | t_ga ms |
|---|---|
| Xsens (base) | −0.82 ± 0.02 |
| GX3-25 | +0.10 ± 0.02 |
| GX3-35 | +0.01 ± 0.02 |
| T265 IMU | −6.74 ± 0.02 |
- Rotations are in
cam_imu.jsonas log(R) rotation vectors in rad. - The T265 IMU has by far the largest
t_ga. The other three stay under 1 ms.
Over 15 recordings (results page,
the “ELP RS stereo + two IMUs (IMUs only)” rows). These runs use a different
setup: the ELP pair only, the T265 IMU as base IMU, the Xsens as extra IMU, and
the Kalibr IMU model. So they are not the same quantities as the tables above;
the Xsens t_ga, for example, is −0.47 ms there against −0.82 ms here. sd is
the standard deviation over the 15 recordings.
| IMU | Da diagonal x / y / z, median | t_ga ms, median |
|---|---|---|
| T265 IMU | 1.01989 / 1.01905 / 1.01668 | −6.697 (sd 0.135) |
| Xsens | 0.99997 / 0.99932 / 1.00093 | −0.466 (sd 0.237) |
- T265: the
Dadiagonal sits 1.7 to 2.0 % above 1 on every axis. - Xsens: stays within 0.1 % of 1.
- Why this table: one run says little; the spread over many recordings shows how repeatable a value is.
6. Leave state_hz alone
Section titled “6. Leave state_hz alone”solve.state_hz sets how many pose, velocity and bias states per second the
solve keeps. It defaults to 30 Hz; our config does not set it. In our tests
on several rigs:
- Fast motion: at low rates the states undersample the motion. The
reprojection error, lever arm and
tddrift, and settle around 30 Hz. - Slow, smooth motion: the rate does not matter.
- Not affected either way: stereo baseline, IMU intrinsics and biases.
So:
- Leave it at 30 unless your motion is slow and you want the speed.
- Check it first if
tdor the lever arm look wrong on a fast-moving rig.
Batch: many recordings from one config
Section titled “Batch: many recordings from one config”To calibrate many recordings at once, list them under input.bags instead of a
single path:
input: format: ros1 bags: - 4imus_4cams_01.bag - 4imus_4cams_02.bag - 4imus_4cams_03.bag- Output: each bag gets its own
bag<NN>/folder in the folder ofoutput.path(results/sweepwhen it has none).<NN>is the bag’s trailing number (4imus_4cams_01.baggivesbag01). - Formats: ROS1 bags, ROS2 bags and EuRoC
mav0folders all work. Each inherits the config’s format (input.format:ros1,ros2oreuroc). - Use: a repeatability sweep over many recordings.
The spread of extrinsics and intrinsics across recordings tells you far more about accuracy than any single run’s residual.
Awkward IMUs
Section titled “Awkward IMUs”Two options help with accelerometers you cannot trust.
Gyro-only: calibrate an extra IMU from its gyro alone.
- Solved: extrinsic rotation,
td,Dw, gyro bias. - Pinned: lever arm,
Da,Tg.
input: imus: - {} # base IMU: needs the accel - {gyro_only: true} # extra IMU- Order: entries match
imu_topicsby order. - Not on the base IMU: the base needs its accelerometer for gravity and metric scale.
Estimate t_ga: solve each IMU’s internal delay between gyroscope and
accelerometer. It is on in our config:
solve: estimate_t_ga: trueBoth options target the same failure. A driver that resamples the
accelerometer to the gyro rate adds a timing offset. Without t_ga, that
offset leaks into td.
The T265 is the clearest case:
- IMU: Bosch BMI055, 62.5 Hz accel interpolated to 200 Hz.
- Measured:
t_ga−6.74 ms on recording 01 with the Xsens as base, and −6.697 ms median over 15 recordings with the T265 IMU as base and the Kalibr IMU model (above). - Not a pure shift: the delay changes with the motion, so one
t_gais a compromise.
Details in IMU model. Why it matters, with real numbers: Why calibrate t_ga.
- Step 5: Rolling-shutter calibration explains the ELP readout lines.
- Step 6: Multi-IMU spatial-temporal calibration explains the three extra IMUs.
- Step 7: Digital-twin check tests the solver on a copy of the rig with known truth.
- Step 8: Round-trip validation checks the result. Do not stop at the residual:
mvis roundtrip -c virig_4imus_4cams_cam_imu.yaml