Skip to content

Multi-IMU spatial-temporal calibration

Step 6 of the walkthrough. Here you add more IMUs to the camera–IMU solve.

  • You need: the config from step 4.
  • You get: for every extra IMU, its lever arm and rotation to the base IMU, its time offset td and its t_ga.
  • Check: compare the numbers across recordings (section 3), and name the base IMU when you quote them.

Two terms:

  • Base IMU: the IMU every other sensor is measured against. It sets the clock and carries the trajectory.
  • Aux IMU (auxiliary IMU): any other IMU on the rig. MVIS solves where it sits and how its clock runs against the base.

All IMUs enter one factor graph with the cameras. There is no pairwise step.

The running example is VI-Rig: 4 IMUs + 4 cameras, bag 4imus_4cams_01.

The first entry in input.imu_topics is the base IMU. It is the reference clock for the whole rig and the only IMU that carries the trajectory. Every other entry is an aux IMU, in the order you list it.

In the running example the base is the Xsens (/imu/data):

virig_4imus_4cams_cam_imu.yaml
input:
imu_topics:
- /imu/data # base: Xsens MTi-100
- /gx3_25/data # aux: GX3-25
- /gx3_35/imu/data # aux: GX3-35
- /t265/imu # aux: T265 IMU
  • Pick a good IMU as the base: its errors spread to the whole rig.
  • Say which base you used when you quote a number. A td or a lever means “against this base”.

2. Turn on the aux spatial and temporal terms

Section titled “2. Turn on the aux spatial and temporal terms”

These keys in the step-4 config control the aux IMUs:

virig_4imus_4cams_cam_imu.yaml
estimate:
imu:
extrinsics: true # lever + rotation of each aux IMU to the base
td: true # time offset of each aux IMU to the base
solve:
estimate_t_ga: true # base and aux IMUs
  • estimate.imu.extrinsics: solves each aux IMU’s lever and rotation. Default true.
  • estimate.imu.td: solves each aux IMU’s time offset to the base. Default true.
  • false on either holds that value at its start value instead.
  • solve.estimate_t_ga: t_ga is the delay between an IMU’s accelerometer and its gyroscope. One flag covers the base and the aux IMUs. Default true.
  • IMU intrinsics (scale and misalignment) follow estimate.imu.intrinsics for every IMU, base and aux. Step 4 sets it to dw_da.

Set gyro_only: true on that IMU. The entries in input.imus match input.imu_topics by order:

virig_4imus_4cams_cam_imu.yaml
input:
imu_topics: [/imu/data, /gx3_25/data, /gx3_35/imu/data, /t265/imu]
imus:
- {} # base: needs its accelerometer
- {}
- {}
- {gyro_only: true} # aux calibrated from its gyroscope alone
  • What it solves: rotation, td, Dw (gyroscope scale and misalignment) and gyroscope bias.
  • What it holds fixed: the lever and the accelerometer terms. There is no t_ga.
  • Not on the base: the base always needs its accelerometer. MVIS stops with an error if you set it there.

The running example does not need it: all four IMUs run in full mode. When to prefer it over t_ga: IMU model.

Each aux IMU adds factors to the graph. Two keys thin them out:

KeyDefaultWhat it does
solve.aux_imu_stride3One aux factor per this many base intervals. Overrides aux_imu_factor_hz.
solve.aux_imu_factor_hz0Aux factor rate in Hz. 0 = one per base interval.

Run it as in step 4:

shell
mvis cam-imu -c virig_4imus_4cams_cam_imu.yaml

Bag 01, Xsens base. The values are from the example run (“Solved rig”):

IMUtd to base (ms)t_ga (ms)lever (mm)rotation log(R) (rad)
Xsens (base)0−0.82 ± 0.0200
GX3-25−0.69+0.10 ± 0.02[−122.8, −45.3, −3.1][+2.2502, +2.1849, −0.0279]
GX3-35−2.07+0.01 ± 0.02[−82.3, −46.2, −3.5][+2.2304, +2.2070, −0.0192]
T265 IMU−7.17−6.74 ± 0.02[+54.7, −107.4, −30.3][+1.5724, −0.0054, −0.0113]
  • Frame: lever and rotation place each IMU in the base IMU frame. The rotation is the log(R) rotation vector.
  • td sign: t_imu = t_sensor + td, with t_imu on the base clock.
  • t_ga: the T265 IMU has by far the largest offset (−6.74 ms), the Xsens −0.82 ms. The two MicroStrain IMUs sit near 0.

Repeatability over 15 bags (“Mixed t265-f1 + ELP-L, aux Xsens” on the results page, T265 IMU base, Xsens aux), median and sd:

Xsens to baseMVISKalibr
td (ms)7.240 (sd 0.146)7.260 (sd 0.122)
distance ‖p‖ (mm)124.09 (sd 0.32)114.10 (sd 2.96)
  • td agrees with Kalibr to 0.02 ms.
  • Kalibr inputs: Kalibr got its camera intrinsics and baselines from an MVIS export, and ELP-L’s stamps were shifted by the MVIS td of the same bag before each run. It treats ELP-L as global shutter. So its td and lever are solved given these MVIS inputs, not independently.
  • The lever does not: the two tools are 10 mm apart, and Kalibr spreads about 9 × wider.
  • MVIS column: MVIS with Kalibr’s IMU model, 15 bags. Kalibr has 14 (bag 8 failed).
  • Base swap: this +7.24 ms and bag 01’s −7.17 ms both link the Xsens and the T265 IMU. The sign flips because the base flips.

Full tables: VI-Rig 4 IMUs + 4 cameras results.

4. Sensor numbering when comparing with Kalibr

Section titled “4. Sensor numbering when comparing with Kalibr”

Both tools number sensors in input order, but they name the IMUs differently. Getting this wrong silently compares the wrong pair.

  • MVIS: cameras are cam0, cam1, … in input.cam_topics order. The base IMU, the first input.imu_topics entry, is imu0. Each aux IMU is named after the first part of its topic: /gx3_25/data becomes gx3_25, /t265/imu becomes t265, /imu/data becomes imu. These names appear in cam_imu.json, the calibration YAML and the HTML report.
  • Kalibr: cameras are cam0, cam1, …; imu0 is its reference IMU, the first IMU it is given, and the others are imu1, imu2, … in input order.
  • Same input order, same base: in the runs below both tools get the same topic lists, so imu0 is the same IMU in both. Match the aux IMUs by topic, not by number.

4 IMUs + 3 cameras (virig-4imus-3cams, Xsens base)

SensorBlackflyT265 LT265 RXsens MTi-100GX3-25GX3-35T265 IMU
Kalibrcam0cam1cam2imu0imu1imu2imu3
MVIScam0cam1cam2imu0gx3_25gx3_35t265

4 IMUs + 4 cameras (the bag-01 run above, Xsens base, MVIS only)

SensorT265 LT265 RELP LELP RXsensGX3-25GX3-35T265 IMU
MVIScam0cam1cam2cam3imu0gx3_25gx3_35t265

Mixed t265-f1 + ELP-L, aux Xsens (T265 IMU base)

SensorT265 LELP LT265 IMUXsens
Kalibrcam0cam1imu0imu1
MVIScam0cam1imu0imu

MVIS reads Kalibr camchain and camchain-imucam files and writes camchain-imucam. See Result formats.

Cameras join the same graph. You set them up in step 3:

  • reference_camera: the camera the other cameras’ extrinsics are expressed against.
  • estimate.camera.td: true (camera-only): solves a time offset between cameras that are not synchronised.
  • Per-camera td: in the camera–IMU solve every camera gets its own td to the base IMU. In bag 01 the T265 cameras sit at −5.41 and −7.47 ms, the ELP cameras at −123.25 and −123.30 ms.

Why one joint solve beats separate ones: Why calibrate cameras together. Why extra IMUs pay off: Why calibrate IMUs together.