Skip to content

IMU model

Use this page to see how MVIS models an IMU, which convention to pick, and how to correct raw IMU readings with a calibration result.

MVIS solves the IMU intrinsics rather than taking them from a datasheet. IMU intrinsics are the sensor’s own error terms:

  • Scale and misalignment of the gyroscope and the accelerometer (Dw\mathbf{D}_w, Da\mathbf{D}_a)
  • The rotation between the two triads (the gyro and accel axes are not exactly parallel)
  • G-sensitivity Tg\mathbf{T}_g: specific force leaking into the gyro (optional)
  • Biases bg\mathbf{b}_g, ba\mathbf{b}_a
  • The gyro–accel time offset tgat_{ga} inside one IMU

Symbols follow Notation & glossary. The code is in src/mvis/imu/: imu_model.py holds the shared equations, kalibr.py, rpng.py and calyx.py the three layouts.

Dw\mathbf{D}_w and Da\mathbf{D}_a are correction matrices: they turn a raw reading into a true value. The raw readings are therefore their inverses applied to the truth. With Iω{}^{I}\boldsymbol{\omega} the true angular velocity and Ia{}^{I}\mathbf{a} the true specific force, both in the IMU frame {I}\{I\}:

am(t)=Da−1 aIR⊤ Ia(t−tga)+ba+na\mathbf{a}_m(t) = \mathbf{D}_a^{-1}\, {}^{I}_{a}\mathbf{R}^{\top}\, {}^{I}\mathbf{a}(t - t_{ga}) + \mathbf{b}_a + \mathbf{n}_a ωm(t)=Dw−1 wIR⊤ Iω(t)+Tg Ia(t)+bg+ng\boldsymbol{\omega}_m(t) = \mathbf{D}_w^{-1}\, {}^{I}_{w}\mathbf{R}^{\top}\, {}^{I}\boldsymbol{\omega}(t) + \mathbf{T}_g\, {}^{I}\mathbf{a}(t) + \mathbf{b}_g + \mathbf{n}_g
  • wIR{}^{I}_{w}\mathbf{R} and aIR{}^{I}_{a}\mathbf{R} rotate the gyro triad {w}\{w\} and the accel triad {a}\{a\} into {I}\{I\}. Each convention fixes one of them to the identity.
  • tgat_{ga} is the gyro–accel offset of the accel against the gyro clock.
  • ng\mathbf{n}_g, na\mathbf{n}_a are white noise. The biases follow a random walk, b˙g=nbg\dot{\mathbf{b}}_g = \mathbf{n}_{bg}, b˙a=nba\dot{\mathbf{b}}_a = \mathbf{n}_{ba}. Their densities are the config’s sigma_g, sigma_a, sigma_bg, sigma_ba.

The correction is the exact inverse. The accel comes first, because the gyro’s g-sensitivity term needs the corrected specific force:

Ia(t)=aIR Da(am(t+tga)−ba){}^{I}\mathbf{a}(t) = {}^{I}_{a}\mathbf{R}\, \mathbf{D}_a \big(\mathbf{a}_m(t + t_{ga}) - \mathbf{b}_a\big) Iω(t)=wIR Dw(ωm(t)−bg−Tg Ia(t)){}^{I}\boldsymbol{\omega}(t) = {}^{I}_{w}\mathbf{R}\, \mathbf{D}_w \big(\boldsymbol{\omega}_m(t) - \mathbf{b}_g - \mathbf{T}_g\, {}^{I}\mathbf{a}(t)\big)

This is the correction the IMU factor applies to every sample inside the solve (src/mvis/calibration/factors/aci.py), the ACI3 integration (cite ACI2). The simulator uses the forward model above to make raw readings, so the two are exact inverses without noise.

cam_imu.yaml
estimate:
imu:
model: calyx # kalibr | rpng | calyx (default calyx)

solve.imu_intrinsics_model is the same setting under its low-level name.

The two triads and {I}\{I\} have 3 + 3 + 3 rotational degrees of freedom, but the data only sees the rotation between the triads. So one rotation must be pinned. What is pinned, and the shape of each matrix, is the convention:

ConventionDw\mathbf{D}_wDa\mathbf{D}_aFree rotation{I}\{I\} isParameters
kalibrlower-triangular (6)lower-triangular (6)wIR{}^{I}_{w}\mathbf{R} (3)the accel triad6 + 6 + 3 + 9 (Tg\mathbf{T}_g) = 24
rpngupper-triangular (6)upper-triangular (6)aIR{}^{I}_{a}\mathbf{R} (3)the gyro triad6 + 6 + 3 + 9 = 24
calyx (default)full 3×33\times3 (9)upper-triangular (6)none: the rotation is inside Dw\mathbf{D}_wthe accel triad9 + 6 + 9 = 24

On top of those 24, every convention has the two biases (3 + 3) and tgat_{ga} (1).

Dw=[dw100dw2dw40dw3dw5dw6],Da=[da100da2da40da3da5da6],aIR=I\mathbf{D}_w = \begin{bmatrix} d_{w1} & 0 & 0 \\ d_{w2} & d_{w4} & 0 \\ d_{w3} & d_{w5} & d_{w6} \end{bmatrix}, \quad \mathbf{D}_a = \begin{bmatrix} d_{a1} & 0 & 0 \\ d_{a2} & d_{a4} & 0 \\ d_{a3} & d_{a5} & d_{a6} \end{bmatrix}, \quad {}^{I}_{a}\mathbf{R} = \mathbf{I} Ia=Da(am−ba),Iω=wIR Dw(ωm−bg−Tg Ia){}^{I}\mathbf{a} = \mathbf{D}_a(\mathbf{a}_m - \mathbf{b}_a), \qquad {}^{I}\boldsymbol{\omega} = {}^{I}_{w}\mathbf{R}\,\mathbf{D}_w(\boldsymbol{\omega}_m - \mathbf{b}_g - \mathbf{T}_g\,{}^{I}\mathbf{a})

This is the structure of Kalibr’s scale-misalignment IMU model. Kalibr’s imu.yaml maps onto it as Dw=Mg−1\mathbf{D}_w = \mathbf{M}_g^{-1}, Da=Ma−1\mathbf{D}_a = \mathbf{M}_a^{-1}, wIR=Cgyro_i⊤{}^{I}_{w}\mathbf{R} = \mathbf{C}_{gyro\_i}^{\top} and Tg=A Cgyro_i\mathbf{T}_g = \mathbf{A}\,\mathbf{C}_{gyro\_i} (src/mvis/io/kalibr.py). Pick it when you compare against Kalibr.

Dw=[dw1dw2dw40dw3dw500dw6],Da=[da1da2da40da3da500da6],wIR=I\mathbf{D}_w = \begin{bmatrix} d_{w1} & d_{w2} & d_{w4} \\ 0 & d_{w3} & d_{w5} \\ 0 & 0 & d_{w6} \end{bmatrix}, \quad \mathbf{D}_a = \begin{bmatrix} d_{a1} & d_{a2} & d_{a4} \\ 0 & d_{a3} & d_{a5} \\ 0 & 0 & d_{a6} \end{bmatrix}, \quad {}^{I}_{w}\mathbf{R} = \mathbf{I} Ia=aIR Da(am−ba),Iω=Dw(ωm−bg−Tg Ia){}^{I}\mathbf{a} = {}^{I}_{a}\mathbf{R}\,\mathbf{D}_a(\mathbf{a}_m - \mathbf{b}_a), \qquad {}^{I}\boldsymbol{\omega} = \mathbf{D}_w(\boldsymbol{\omega}_m - \mathbf{b}_g - \mathbf{T}_g\,{}^{I}\mathbf{a})

The convention of the RPNG group’s VINS work (the OpenVINS lineage). Here {I}\{I\} is the gyro triad, so the camera–IMU extrinsics are relative to the gyro axes.

Dw=[dw1dw2dw3dw4dw5dw6dw7dw8dw9],Da upper-triangular as in rpng,wIR=aIR=I\mathbf{D}_w = \begin{bmatrix} d_{w1} & d_{w2} & d_{w3} \\ d_{w4} & d_{w5} & d_{w6} \\ d_{w7} & d_{w8} & d_{w9} \end{bmatrix}, \quad \mathbf{D}_a \text{ upper-triangular as in } \texttt{rpng}, \quad {}^{I}_{w}\mathbf{R} = {}^{I}_{a}\mathbf{R} = \mathbf{I} Ia=Da(am−ba),Iω=Dw(ωm−bg−Tg Ia){}^{I}\mathbf{a} = \mathbf{D}_a(\mathbf{a}_m - \mathbf{b}_a), \qquad {}^{I}\boldsymbol{\omega} = \mathbf{D}_w(\boldsymbol{\omega}_m - \mathbf{b}_g - \mathbf{T}_g\,{}^{I}\mathbf{a})

The full Dw\mathbf{D}_w is linear in its nine entries, so the solve needs no rotation variable. After the solve, MVIS splits it into a rotation and an upper-triangular matrix with a positive diagonal (a QR decomposition):

Dw=wIR  Uw\mathbf{D}_w = {}^{I}_{w}\mathbf{R}\; \mathbf{U}_w

The result reports both: R_wtoI, U_w, and Dw_decomposed with the per-axis gyro scale (Dw_scale, the diagonal of Uw\mathbf{U}_w), the cross-coupling (Dw_shear, its upper off-diagonals) and gyro_misalign_rotvec =Log(wIR)= \mathrm{Log}({}^{I}_{w}\mathbf{R}) in rad. Read the scale from Dw_scale, not from Dw_diag: in calyx the diagonal of Dw\mathbf{D}_w mixes scale with rotation.

  • Same physics, different bookkeeping. All three describe the same sensor. They differ in which frame {I}\{I\} is and where the triad rotation sits.
  • Not a quality lever. Before the 0.1.0 release we checked this on TUM-VI and on the Monado SLAM calibration set MIC (Valve Index, see Datasets), with camera intrinsics pinned or freed. All three recovered the same extrinsics and camera intrinsics. Choose the one that matches what you compare against.
  • Not comparable entry by entry. A kalibr Dw\mathbf{D}_w and an rpng Dw\mathbf{D}_w of the same IMU differ. Before concluding that two calibrations disagree, check they used the same convention.
  • Default staging differs. The default estimate.imu.intrinsics: dw_da frees Dw\mathbf{D}_w and Da\mathbf{D}_a only. In calyx that already includes the triad rotation (it lives in Dw\mathbf{D}_w). In kalibr and rpng the rotation stays at the identity until you choose dw_da_r.
estimate.imu.intrinsicsFrees
pinned (or false)nothing
dw_da (default)Dw\mathbf{D}_w, Da\mathbf{D}_a
dw_da_rDw\mathbf{D}_w, Da\mathbf{D}_a, the triad rotation (kalibr, rpng; same as dw_da in calyx)
full (or true)the above plus Tg\mathbf{T}_g

Freed blocks carry weak priors around their start values: the identity for Dw\mathbf{D}_w, Da\mathbf{D}_a and the rotation, zero for Tg\mathbf{T}_g (sigma_dw_da 0.1, sigma_imu_R 0.1 rad, sigma_tg 0.05).

mvis cam-imu writes a calibration YAML next to its JSON summary (for cam_imu.json, the file is cam_imu_calib.yaml). Each entry of its imus: list has:

KeyMeaning
name, role, topicIMU name, base or aux, and its topic when known
modelkalibr, rpng or calyx
Dw, Dathe correction matrices, 3×33\times3
imu_intr_xithe 24 intrinsics as an offset from the identity, in the convention’s order; it also carries Tg\mathbf{T}_g and the triad rotation
b_g, b_abiases, rad/s and m/s²
t_ga, t_ga_sigmagyro–accel offset and its 1-sigma, s
Dw_diag, Da_diagthe diagonals, for a quick look only; in calyx Dw_diag is not a scale
Dw_decomposedcalyx only (base and aux): scale, shear and misalignment rotation
R_wtoI, U_wcalyx, base IMU only: the split above
tdaux only: IMU–IMU time offset, s
p_IsinI_maux only: lever arm to the base IMU, m
gyro_onlyaux only, present when the IMU was calibrated gyro-only

The layout of imu_intr_xi:

ConventionDw\mathbf{D}_wDa\mathbf{D}_aTg\mathbf{T}_g (column-major)rotation vector
kalibr[0:6][6:12][12:21][21:24], wIR{}^{I}_{w}\mathbf{R}
rpng[0:6][6:12][12:21][21:24], aIR{}^{I}_{a}\mathbf{R}
calyx[0:9][9:15][15:24]none

mvis.recalib.load_rig_calib reads any convention and applies the inverse model above:

from mvis.recalib import load_rig_calib
imu = load_rig_calib("cam_imu_calib.yaml").base_imu
w, a = imu.intr.correct(w_m, a_m, imu.b_g, imu.b_a) # true omega, true specific force in {I}

Read the YAML with yaml.safe_load and pass one imus: entry. The Dw and Da matrices come straight from the file; Tg\mathbf{T}_g and the rotation come from imu_intr_xi. w_m and a_m are length-3 arrays of one sample, with the accel already shifted by tgat_{ga} (below).

kalibr
import numpy as np
from scipy.spatial.transform import Rotation
def correct_kalibr(w_m, a_m, imu):
xi = np.asarray(imu["imu_intr_xi"])
Dw, Da = np.asarray(imu["Dw"]), np.asarray(imu["Da"]) # lower-triangular
Tg = xi[12:21].reshape(3, 3, order="F")
R_wtoI = Rotation.from_rotvec(xi[21:24]).as_matrix()
a = Da @ (a_m - imu["b_a"]) # accel triad = {I}
w = R_wtoI @ Dw @ (w_m - imu["b_g"] - Tg @ a)
return w, a
rpng
import numpy as np
from scipy.spatial.transform import Rotation
def correct_rpng(w_m, a_m, imu):
xi = np.asarray(imu["imu_intr_xi"])
Dw, Da = np.asarray(imu["Dw"]), np.asarray(imu["Da"]) # upper-triangular
Tg = xi[12:21].reshape(3, 3, order="F")
R_atoI = Rotation.from_rotvec(xi[21:24]).as_matrix()
a = R_atoI @ Da @ (a_m - imu["b_a"])
w = Dw @ (w_m - imu["b_g"] - Tg @ a) # gyro triad = {I}
return w, a
calyx
import numpy as np
def correct_calyx(w_m, a_m, imu):
xi = np.asarray(imu["imu_intr_xi"])
Dw, Da = np.asarray(imu["Dw"]), np.asarray(imu["Da"]) # full, upper-triangular
Tg = xi[15:24].reshape(3, 3, order="F")
a = Da @ (a_m - imu["b_a"]) # accel triad = {I}
w = Dw @ (w_m - imu["b_g"] - Tg @ a) # Dw already holds the triad rotation
return w, a

We checked all three against ImuModel.correct in MVIS on random intrinsics; they agree to machine precision.

Do this first, on the whole recording: re-sample the accel channel at each stamp plus tgat_{ga}, so that gyro and accel describe the same instant.

import numpy as np
def shift_accel(t, acc, t_ga):
"""t: (N,) stamps in s, sorted; acc: (N, 3) raw accel. Returns acc at t + t_ga."""
return np.stack([np.interp(t + t_ga, t, acc[:, k]) for k in range(3)], axis=1)

mvis apply-calib writes a corrected copy of a ROS1 bag (src/mvis/recalib/apply.py). On each IMU message, in this order:

  1. t_ga: the accel is linearly re-interpolated at header stamp + tgat_{ga} (--no-t-ga skips it).
  2. Intrinsics and bias: the inverse model above subtracts the saved bg\mathbf{b}_g, ba\mathbf{b}_a and removes Dw\mathbf{D}_w, Da\mathbf{D}_a, Tg\mathbf{T}_g and the triad rotation. --keep-bias leaves the biases in; --no-imu-intrinsics removes only the biases.
  3. td: the header stamp becomes stamp + td (--no-td keeps the original).

An auxiliary IMU entry without imu_intr_xi is read with identity intrinsics, so only its saved bias (if any) and its stamps change.

tgat_{ga} is the time offset between the gyroscope and accelerometer channels inside one IMU.

Many drivers publish gyro and accel together under one stamp, yet the two channels are sampled at different rates and instants. MVIS takes the shared stamp as the gyro clock and models the accel stored at tt as sampled at t−tgat - t_{ga}.

cam_imu.yaml
solve:
estimate_t_ga: true # default in camera–IMU runs
sigma_t_ga: 0.010 # weak prior, s
  • In the solve: the IMU factor corrects to first order, am(t+tga)≈am(t)+tga a˙m(t)\mathbf{a}_m(t + t_{ga}) \approx \mathbf{a}_m(t) + t_{ga}\,\dot{\mathbf{a}}_m(t), with a˙m\dot{\mathbf{a}}_m from finite differences of the raw accel. That adds one Jacobian column; through Tg\mathbf{T}_g it also reaches the gyro row. When tgat_{ga} moves more than 3 ms from where the factors were integrated, they are integrated again (tga_relin_threshold_s).
  • Reported in the calibration YAML, the cam_imu.json summary (t_ga_ms) and the report.
  • Corrected by mvis apply-calib, as above.
  • Checked by mvis roundtrip: after correction a re-calibration should find tgat_{ga} near zero.

Why it matters: a tool without tgat_{ga} puts part of that delay into the camera tdt_d. On real devices a single tgat_{ga} can also depend on the motion. Both are shown in Why calibrate t_ga.

The other answer to an untrustworthy accelerometer is to skip it:

cam_imu.yaml
input:
imus:
- {} # base IMU
- {gyro_only: true} # auxiliary IMU

What a gyro-only IMU solves and pins:

  • Solved: extrinsic rotation, td (here the IMU-to-IMU time offset), Dw\mathbf{D}_w and the gyro bias.
  • Pinned: the lever arm (the translation between the two IMUs), Da\mathbf{D}_a and Tg\mathbf{T}_g.

It is not allowed on the base IMU: the base IMU needs its accelerometer for gravity and metric scale. The result marks each such IMU with gyro_only: true.

We built it for the Intel T265 (Bosch BMI055), whose 62.5 Hz accel the driver interpolates to 200 Hz. We calibrated, corrected the bag and calibrated again. Without gyro-only, the T265’s td moved from +8.6 ms to −9.0 ms between the two runs. With gyro-only it moved from +0.9 ms to +0.4 ms.

Both address an accelerometer that is out of step with its gyro:

OptionWhat it doesPrefer it for
estimate_t_gaMeasures the offsetThe base IMU, which must use its accel; the value itself is worth having
gyro_onlyAvoids the accelerometerAn auxiliary IMU whose accel adds nothing you need

Each term needs the right motion:

  • Scale and misalignment need rotation and acceleration on every axis.
  • The triad rotation needs rotation about several axes while the accelerometer sees gravity from several directions.
  • G-sensitivity Tg\mathbf{T}_g needs rotation and strong linear acceleration at the same time. Most hand-held motions do not produce that, which is why the default leaves it out.
  • tgat_{ga} needs the specific force to change quickly, so that a time shift changes the reading.

When a freed term returns near its start value with a wide covariance, it was not observed. See Observability and Degeneracy.