Skip to content

Camera–IMU calibration

Use this page to solve cameras and IMUs in one graph. You need the recording from step 2 and the camera YAML from step 3. You get cam_imu.json and a calibration YAML.

The running example is recording 01 of our VI-Rig with 4 IMUs and 4 cameras (dataset page):

  • Cameras: a T265 fisheye pair (global shutter) and an ELP stereo pair (rolling shutter).
  • IMUs: Xsens MTi-100, MicroStrain GX3-25 and GX3-35, and the T265 IMU.

The solve estimates, all at once:

  • Extrinsics: where each camera and IMU sits relative to the base IMU (lever arm and rotation).
  • Time offset td: the clock delay of each camera and each extra IMU to the base IMU.
  • IMU intrinsics: scale and misalignment of each gyroscope (Dw) and accelerometer (Da).
  • t_ga: the delay between an IMU’s own gyroscope and accelerometer samples.
  • Camera intrinsics: refined from the step 3 result, not held fixed.
  • Biases of every gyroscope and accelerometer.

Step 3 (Camera-only calibration) writes a camera YAML. Name it in the cam-IMU config:

virig_4imus_4cams_cam_imu.yaml
camera_input:
path: camera_prior.yaml
  • What it gives: the starting intrinsics of each camera (the prior).
  • One file, in cam_topics order: join the T265 and ELP results of step 3. MVIS sorts the cameras by id and matches them to cam_topics by order. So here the T265 fisheyes are id 0 and 1, the ELP left and right 2 and 3.
  • Our example’s prior: camera_prior.yaml. It joins earlier camera-only results and uses equidistant for the T265, which gives the same result as kb4 on this lens (step 3). So the report lists the T265 cameras as equidistant.
  • What happens next: the cam-IMU solve refines the intrinsics (estimate.camera.intrinsics: true). It solves the extrinsics and time offsets itself; the prior’s extrinsics are not used.

This is the config of the VI-Rig 4 IMUs + 4 cameras example, recording 01. The highlighted lines matter for this step.

virig_4imus_4cams_cam_imu.yaml
input:
format: ros1
path: 4imus_4cams_01.bag
cam_topics:
- /t265/fisheye1/image_raw
- /t265/fisheye2/image_raw
- /elp/split_sync_image_node/left/image_raw
- /elp/split_sync_image_node/right/image_raw
imu_topics:
- /imu/data
- /gx3_25/data
- /gx3_35/imu/data
- /t265/imu
window_mode: motion
segment_length: 80.0
board:
board_type: apriltag
rows: 6
cols: 6
tag_size: 0.088
tag_spacing: 0.3
tag_family: t36h11
heights_file: board_shape_4imus_4cams.yaml
camera_input:
path: camera_prior.yaml
cameras:
- {}
- factor_hz: 4.0
- estimate_readout: true
factor_hz: 4.0
- estimate_readout: true
factor_hz: 4.0
solve:
max_iterations: 40
corner_subsample: two_anti
estimate_t_ga: true
sigma_t_ga: 0.01
vision_pose: imu
sigma_g: 0.00065
sigma_a: 0.0026
sigma_bg: 0.001
sigma_ba: 0.002
estimate:
board_points: true
camera:
intrinsics: true
tr: false
td: true
extrinsics: true
imu:
intrinsics: dw_da
model: calyx
extrinsics: true
td: true
output:
path: cam_imu.json

What the highlighted lines do:

  • input: a ROS1 bag. The first imu_topics entry (/imu/data, the Xsens) is the base IMU. Every other sensor is solved relative to it.
  • window_mode: motion: MVIS uses the segment_length seconds (here 80 s) with the most IMU motion.
  • board.heights_file + estimate.board_points: true: solve the board shape too, starting from a measured shape. board_shape_4imus_4cams.yaml ships with MVIS in examples/regression/mvis_elp/. Our board is not flat; a flat solve shifts the gyro scale and the lever arm (why).
  • solve.corner_subsample: two_anti: use two opposite corners of each tag (top-left and bottom-right). Our config sets it for speed. The cam-IMU default is all (every corner).
  • solve.estimate_t_ga: true, sigma_t_ga: 0.01: solve t_ga for each IMU. Keep this on.
  • solve.vision_pose: imu: moves the pose to each corner’s capture time with the raw base-IMU samples. It is the default.
  • sigma_g, sigma_a, sigma_bg, sigma_ba: gyro and accel noise, and their bias random walk.
  • estimate.camera: intrinsics free, td and extrinsics solved.
  • estimate.imu: IMU intrinsics dw_da with the calyx model (see section 3), plus extrinsics and td of the extra IMUs.

The other lines belong to later steps:

  • camera_input.cameras: one entry per camera, in cam_topics order. The three cameras after the first enter the graph at 4 frames per second (factor_hz), which keeps the solve small. The first camera keeps the cam-IMU default of 10 Hz.
  • ELP readout lines (estimate_readout, tr): estimate.camera.tr: false is the rig-wide default and holds the readout fixed. The per-camera estimate_readout: true on the two ELP entries overrides it, so the run solves the ELP readout and keeps the T265 (global shutter) at zero. Details in step 5, Rolling-shutter calibration.
  • The three extra IMUs: step 6, Multi-IMU spatial-temporal calibration.

For your own rig, change the topics, the board and the prior path. Keep the rest as a starting point.

estimate.imu.intrinsics sets how much of each IMU model we solve:

ValueWhat is solved
pinnednothing: Dw and Da stay at their start values
dw_dascale and misalignment of gyro (Dw) and accel (Da). Our setting, and the default
dw_da_ras dw_da, plus the gyro–accel rotation. Same as dw_da for calyx, which keeps that rotation inside Dw
fullas dw_da_r, plus Tg (how linear acceleration leaks into the gyro)
  • A bool also works: true means full, false means pinned.
  • estimate.imu.model picks the model family: calyx (default), kalibr or rpng.
  • Priors: freed terms get a weak prior. solve.sigma_dw_da defaults to 0.1, solve.sigma_tg to 0.05.
virig_4imus_4cams_cam_imu.yaml
estimate:
imu:
intrinsics: dw_da # pinned | dw_da | dw_da_r | full
model: calyx # calyx | kalibr | rpng

What the terms mean: IMU model. Why it pays off: Why calibrate the IMU.

shell
mvis cam-imu -c virig_4imus_4cams_cam_imu.yaml
  • Other bag: --bag <file> replaces the config’s source with that ROS1 bag.
  • Time: 22 min in our run (MVIS 0.1.0, 2026-11-12), on 16 CPU cores with one thread for the linear-algebra libraries, on a machine under heavy load. Expect less on your machine. About 12 min of that is AprilTag detection on the four cameras; the solve stages take about 11 min, of which the 6 iterations take about 7 min (solve_s in the report’s Timing section).
  • Output: in the folder of output.path:
    • cam_imu.json: a summary of the solve.
    • cam_imu_calib.yaml: the full calibration, the file to deploy.
    • cam_imu_camchain-imucam.yaml: the same rig in Kalibr’s camchain format.
    • cam_imu_board_shape.yaml: the solved board shape (when estimate.board_points is on).
    • cam_imu_report.html: the report.
    • cam_imu_views.json: the data of the report’s 3D views.

Recording 01 gives the values below. The full table and a 3D view of the rig are on the example page.

The report. Every run writes cam_imu_report.html next to cam_imu.json. It has these sections:

  • Summary: status, reprojection median and P95, χ²/dof of the camera and IMU terms with a grade each, and the base IMU’s t_ga. χ²/dof near 1 means the residuals match the noise you set; well below 1 (0.13 here) means the noise settings are conservative.
  • 3D Geometry: the solved rig path around the board as a player, and the rig to scale.
  • Cameras: per camera, intrinsics, distortion, lever arm and rotation (log(R), rad) with their σ, td, readout time, and the corner accounting (corners used, past the lens fold, dropped as outliers).
  • Board Shape: the solved corner heights as a table and a map, with the median σ of a height. Shown because estimate.board_points is on.
  • Reprojection Analysis: error by frame, by tag and over the board, per camera.
  • Motion & Excitation: the angular velocity of the solved path, which shows how well the motion constrains the lever arms and td.
  • Base IMU, Aux IMUs, IMU Residuals: biases and their drift, Dw, Da, Tg and t_ga of each IMU; each extra IMU’s pose and td relative to the base IMU. The block is titled “IMU intrinsics (full)” whatever level you solve; with dw_da, as here, Tg stays at zero.
  • Timing, Result files, Configuration: where the time went, the files the run wrote, and the config it read.

This is the report of the recording 01 run, unchanged except that local folder names are cut down to file names:

Open the report full screen (HTML, 12.8 MB).

Fit and geometry:

ValueRecording 01
Reprojection median, all cameras0.25 px (rms 0.52, p95 0.68)
cam0–cam1 baseline (T265 pair)63.7 mm
cam0 td−5.41 ms
cam0 lever arm130.8 mm
Board shape9.0 mm peak to peak, 2.2 mm RMS (72 of 144 corners seen)
  • cam0: T265 fisheye 1. cam1 is T265 fisheye 2, cam2 and cam3 are ELP left and right.
  • 72 of 144 corners: with two_anti, only two corners of each tag enter the solve. The other heights stay at the measured shape from heights_file.
  • Reprojection shows how well the model fits the corners. It does not show how accurate the model is.

Base IMU (Xsens) intrinsics, diagonal terms:

xyz
Dw (gyro)0.998130.997681.00090
Da (accel)0.999940.996421.00124

t_ga per IMU:

IMUt_ga ms
Xsens (base)−0.82 ± 0.02
GX3-25+0.10 ± 0.02
GX3-35+0.01 ± 0.02
T265 IMU−6.74 ± 0.02
  • Rotations are in cam_imu.json as log(R) rotation vectors in rad.
  • The T265 IMU has by far the largest t_ga. The other three stay under 1 ms.

Over 15 recordings (results page, the “ELP RS stereo + two IMUs (IMUs only)” rows). These runs use a different setup: the ELP pair only, the T265 IMU as base IMU, the Xsens as extra IMU, and the Kalibr IMU model. So they are not the same quantities as the tables above; the Xsens t_ga, for example, is −0.47 ms there against −0.82 ms here. sd is the standard deviation over the 15 recordings.

IMUDa diagonal x / y / z, mediant_ga ms, median
T265 IMU1.01989 / 1.01905 / 1.01668−6.697 (sd 0.135)
Xsens0.99997 / 0.99932 / 1.00093−0.466 (sd 0.237)
  • T265: the Da diagonal sits 1.7 to 2.0 % above 1 on every axis.
  • Xsens: stays within 0.1 % of 1.
  • Why this table: one run says little; the spread over many recordings shows how repeatable a value is.

solve.state_hz sets how many pose, velocity and bias states per second the solve keeps. It defaults to 30 Hz; our config does not set it. In our tests on several rigs:

  • Fast motion: at low rates the states undersample the motion. The reprojection error, lever arm and td drift, and settle around 30 Hz.
  • Slow, smooth motion: the rate does not matter.
  • Not affected either way: stereo baseline, IMU intrinsics and biases.

So:

  • Leave it at 30 unless your motion is slow and you want the speed.
  • Check it first if td or the lever arm look wrong on a fast-moving rig.

To calibrate many recordings at once, list them under input.bags instead of a single path:

virig_4imus_4cams_cam_imu.yaml
input:
format: ros1
bags:
- 4imus_4cams_01.bag
- 4imus_4cams_02.bag
- 4imus_4cams_03.bag
  • Output: each bag gets its own bag<NN>/ folder in the folder of output.path (results/sweep when it has none). <NN> is the bag’s trailing number (4imus_4cams_01.bag gives bag01).
  • Formats: ROS1 bags, ROS2 bags and EuRoC mav0 folders all work. Each inherits the config’s format (input.format: ros1, ros2 or euroc).
  • Use: a repeatability sweep over many recordings.

The spread of extrinsics and intrinsics across recordings tells you far more about accuracy than any single run’s residual.

Two options help with accelerometers you cannot trust.

Gyro-only: calibrate an extra IMU from its gyro alone.

  • Solved: extrinsic rotation, td, Dw, gyro bias.
  • Pinned: lever arm, Da, Tg.
virig_4imus_4cams_cam_imu.yaml
input:
imus:
- {} # base IMU: needs the accel
- {gyro_only: true} # extra IMU
  • Order: entries match imu_topics by order.
  • Not on the base IMU: the base needs its accelerometer for gravity and metric scale.

Estimate t_ga: solve each IMU’s internal delay between gyroscope and accelerometer. It is on in our config:

virig_4imus_4cams_cam_imu.yaml
solve:
estimate_t_ga: true

Both options target the same failure. A driver that resamples the accelerometer to the gyro rate adds a timing offset. Without t_ga, that offset leaks into td.

The T265 is the clearest case:

  • IMU: Bosch BMI055, 62.5 Hz accel interpolated to 200 Hz.
  • Measured: t_ga −6.74 ms on recording 01 with the Xsens as base, and −6.697 ms median over 15 recordings with the T265 IMU as base and the Kalibr IMU model (above).
  • Not a pure shift: the delay changes with the motion, so one t_ga is a compromise.

Details in IMU model. Why it matters, with real numbers: Why calibrate t_ga.

shell
mvis roundtrip -c virig_4imus_4cams_cam_imu.yaml