Skip to content

Base IMU factor

Use this page to see how MVIS turns the base IMU’s raw readings into a factor between two trajectory states, with the IMU’s own errors solved inside it.

  • Base IMU: the first IMU in the config. Its clock is the reference clock and its pose is the trajectory.
  • State: the IMU pose, velocity and biases at one time tkt_k. States sit at solve.state_hz (30 Hz by default) on the first camera’s frames.
  • Preintegration: the IMU readings between two states, combined into one relative rotation, velocity change and position change.

Symbols follow Notation & glossary. The IMU conventions (calyx, kalibr, rpng) are on IMU model.

The gyroscope gives ωm\boldsymbol{\omega}_m and the accelerometer am\mathbf{a}_m. MVIS corrects each reading before it integrates it:

Ia=aIR  Da (am−ba){}^{I}\mathbf{a} = {}^{I}_{a}\mathbf{R}\;\mathbf{D}_a\,\big(\mathbf{a}_m - \mathbf{b}_a\big) Iω=wIR  Dw (ωm−bg−Tg Ia){}^{I}\boldsymbol{\omega} = {}^{I}_{w}\mathbf{R}\;\mathbf{D}_w\,\big(\boldsymbol{\omega}_m - \mathbf{b}_g - \mathbf{T}_g\,{}^{I}\mathbf{a}\big)
  • Dw\mathbf{D}_w, Da\mathbf{D}_a: correction matrices. Their diagonal fixes the scale of each axis; the rest fixes misalignment (axes not quite at right angles). They multiply the raw reading, so a diagonal of 0.998 means the raw axis read 0.2 % high.
  • Tg\mathbf{T}_g: g-sensitivity, the part of the gyro reading caused by linear acceleration.
  • bg\mathbf{b}_g, ba\mathbf{b}_a: the biases, solved at every state.
  • wIR{}^{I}_{w}\mathbf{R}, aIR{}^{I}_{a}\mathbf{R}: the gyro and accelerometer triad rotations into the IMU frame. Each convention frees at most one of them:
ConventionDw\mathbf{D}_wDa\mathbf{D}_aFree rotation{I}\{I\} isIntrinsic parameters
calyx (default)full 3×3 (9)upper-triangular (6)none; folded into Dw\mathbf{D}_wthe accel triad9 + 6 + 9 (Tg\mathbf{T}_g) = 24
kalibrlower-triangular (6)lower-triangular (6)wIR{}^{I}_{w}\mathbf{R} (3)the accel triad6 + 6 + 9 + 3 = 24
rpngupper-triangular (6)upper-triangular (6)aIR{}^{I}_{a}\mathbf{R} (3)the gyro triad6 + 6 + 9 + 3 = 24

All three give the solver one 24-value intrinsics variable xin\mathbf{x}_{in}. It starts at identity (no correction).

An IMU slower than 200 Hz is first upsampled (cubic, the real samples kept) before integration (solve.imu_min_rate_hz).

Between two states at tkt_k and tk+1t_{k+1} (Δt=tk+1−tk\Delta t = t_{k+1} - t_k), the corrected readings give the motion in the frame of the first state:

ΔRk=Ik+1IkR,Δvk=∫tktk+1IτIkR  Iaτ dτ,Δpk=∫tktk+1 ⁣ ⁣∫tksIτIkR  Iaτ dτ ds\Delta\mathbf{R}_{k} = {}^{I_k}_{I_{k+1}}\mathbf{R}, \qquad \Delta\mathbf{v}_{k} = \int_{t_k}^{t_{k+1}} {}^{I_k}_{I_\tau}\mathbf{R}\;{}^{I}\mathbf{a}_\tau\,d\tau, \qquad \Delta\mathbf{p}_{k} = \int_{t_k}^{t_{k+1}}\!\!\int_{t_k}^{s} {}^{I_k}_{I_\tau}\mathbf{R}\;{}^{I}\mathbf{a}_\tau\,d\tau\,ds

MVIS evaluates these with ACI3: analytic combined IMU integration (ACI2, Yang et al., ICRA 2020) extended with the IMU intrinsics in the MVIS paper. Between two samples it takes the midpoint of the two corrected readings as constant and integrates the rotation and the rotated acceleration in closed form. The same pass gives:

  • the Jacobians of ΔR\Delta\mathbf{R}, Δv\Delta\mathbf{v}, Δp\Delta\mathbf{p} with respect to the biases, the 24 intrinsics and t_ga;
  • the 15×15 covariance Pk\mathbf{P}_k, from the noise densities (solve.sigma_g, sigma_a) and the bias random-walk densities (sigma_bg, sigma_ba).

The integration runs once, at the biases and intrinsics of the start point. Small changes after that are applied through the Jacobians. A detection gap is bridged by one factor over the whole gap.

Many drivers publish gyro and accelerometer on one stamp, though the two were sampled at different instants. MVIS takes the stamp as gyro time. The accelerometer value that belongs to gyro time tt is the one stamped t+tgat + t_{ga}. Over one integration step from sample 0 to sample 1, to first order:

aˉm=12(am,0+am,1)+tga a˙m,a˙m=am,1−am,0δt\bar{\mathbf{a}}_m = \tfrac12\big(\mathbf{a}_{m,0} + \mathbf{a}_{m,1}\big) + t_{ga}\,\dot{\mathbf{a}}_m, \qquad \dot{\mathbf{a}}_m = \frac{\mathbf{a}_{m,1} - \mathbf{a}_{m,0}}{\delta t}
  • One more Jacobian column: ∂aˉm/∂tga=a˙m\partial\bar{\mathbf{a}}_m/\partial t_{ga} = \dot{\mathbf{a}}_m, carried through Da\mathbf{D}_a and, through Tg\mathbf{T}_g, into the gyro.
  • Re-integration: after the first two iterations, when t_ga has moved more than 3 ms from the value the factors were integrated at, MVIS integrates that IMU again with its accelerometer resampled at t+tgat + t_{ga} (solve.tga_relin_threshold_s, default 0.003 s). The linear term then covers only what is left.
  • Prior: a weak zero-mean prior, 10 ms (solve.sigma_t_ga).

estimate_t_ga is on by default in camera–IMU runs. What the offset is and when it matters: Why calibrate t_ga.

The factor links state kk (pose IkGT{}^{G}_{I_k}\mathbf{T}, velocity GvIk{}^{G}\mathbf{v}_{I_k}, biases) to state k+1k+1. It predicts the measured preintegration from the two states and compares.

Write δβ\delta\boldsymbol{\beta} for the change since integration: δbg\delta\mathbf{b}_g, δba\delta\mathbf{b}_a, δxin\delta\mathbf{x}_{in} and δtga\delta t_{ga}. First-order corrections, built from the Jacobians above:

δθ=JR δβ,δv=Jv δβ,δp=Jp δβ\delta\boldsymbol\theta = \mathbf{J}_{R}\,\delta\boldsymbol\beta, \qquad \delta\mathbf{v} = \mathbf{J}_{v}\,\delta\boldsymbol\beta, \qquad \delta\mathbf{p} = \mathbf{J}_{p}\,\delta\boldsymbol\beta

The 15 residual rows are:

rθ=Log ⁣(ΔRk⊤  IkGR⊤ Ik+1GR  Exp(δθ)⊤)\mathbf{r}_\theta = \mathrm{Log}\!\Big(\Delta\mathbf{R}_k^\top\;{}^{G}_{I_k}\mathbf{R}^\top\,{}^{G}_{I_{k+1}}\mathbf{R}\;\mathrm{Exp}(\delta\boldsymbol\theta)^\top\Big) rp=IkGR⊤(GpIk+1−GpIk−GvIk Δt−12 Gg Δt2)−δp−Δpk\mathbf{r}_p = {}^{G}_{I_k}\mathbf{R}^\top\Big({}^{G}\mathbf{p}_{I_{k+1}} - {}^{G}\mathbf{p}_{I_k} - {}^{G}\mathbf{v}_{I_k}\,\Delta t - \tfrac12\,{}^{G}\mathbf{g}\,\Delta t^2\Big) - \delta\mathbf{p} - \Delta\mathbf{p}_k rv=IkGR⊤(GvIk+1−GvIk−Gg Δt)−δv−Δvk\mathbf{r}_v = {}^{G}_{I_k}\mathbf{R}^\top\Big({}^{G}\mathbf{v}_{I_{k+1}} - {}^{G}\mathbf{v}_{I_k} - {}^{G}\mathbf{g}\,\Delta t\Big) - \delta\mathbf{v} - \Delta\mathbf{v}_k rb=[bg,k+1−bg,kba,k+1−ba,k]\mathbf{r}_{b} = \begin{bmatrix}\mathbf{b}_{g,k+1} - \mathbf{b}_{g,k}\\ \mathbf{b}_{a,k+1} - \mathbf{b}_{a,k}\end{bmatrix}

The cost is ∥r∥Pk2\|\mathbf{r}\|^2_{\mathbf{P}_k} with r=[rθ; rp; rv; rb]\mathbf{r} = [\mathbf{r}_\theta;\ \mathbf{r}_p;\ \mathbf{r}_v;\ \mathbf{r}_b].

  • Bias random walk: rb\mathbf{r}_b is the random-walk term. Biases may drift between states, by as much as the random-walk densities allow. There is no separate bias factor.
  • No robust loss: the IMU residual is plain least squares. Only the reprojection factor uses Huber.
  • The covariance follows the correction: Pk\mathbf{P}_k is rotated by the current δθ\delta\boldsymbol\theta before it weights the residual.

Every block is analytic and checked against finite differences in the test suite.

VariableRows it touchesComes from
pose kkrθ\mathbf{r}_\theta, rp\mathbf{r}_p, rv\mathbf{r}_vthe predicted relative motion, rotated into frame kk
pose k+1k+1rθ\mathbf{r}_\theta, rp\mathbf{r}_pthe same, at the end point
velocity kk, k+1k+1rp\mathbf{r}_p, rv\mathbf{r}_vIkGR⊤{}^{G}_{I_k}\mathbf{R}^\top, times Δt\Delta t in rp\mathbf{r}_p
biases kkallthe bias Jacobians of the integration; identity in rb\mathbf{r}_b
biases k+1k+1rb\mathbf{r}_bidentity
intrinsics xin\mathbf{x}_{in}rθ\mathbf{r}_\theta, rp\mathbf{r}_p, rv\mathbf{r}_vthe intrinsic Jacobians of the integration
t_garθ\mathbf{r}_\theta, rp\mathbf{r}_p, rv\mathbf{r}_vthe t_ga column of the integration

estimate.imu.intrinsics frees blocks of xin\mathbf{x}_{in}. A pinned block gets a very tight prior at its start value; a freed block gets a weak one, so a direction the motion does not excite stays near its start instead of drifting.

SettingFreeWeak prior on the free terms
pinned (or false)nothing—
dw_da (default)Dw\mathbf{D}_w, Da\mathbf{D}_a0.1 (solve.sigma_dw_da)
dw_da_ras dw_da, plus the free rotation (kalibr, rpng; same as dw_da under calyx, which has no separate rotation)0.1 (solve.sigma_imu_R)
full (or true)as dw_da_r, plus Tg\mathbf{T}_g0.05 (solve.sigma_tg)

The same model inside the reprojection factor

Section titled “The same model inside the reprojection factor”

The reprojection factor integrates the same corrected readings from an anchor state to each corner’s capture time (Camera–IMU calibration math). So Dw\mathbf{D}_w, Da\mathbf{D}_a, the biases and t_ga are fitted by both factors, with one consistent model.

  • Rotate about all three axes. Each column of Dw\mathbf{D}_w needs rotation about its axis.
  • Point every axis at gravity at some time. Gravity is the largest specific force you have. Tilting the rig through many orientations separates Da\mathbf{D}_a from ba\mathbf{b}_a.
  • Change the acceleration briskly. t_ga appears only through a˙m\dot{\mathbf{a}}_m; smooth, steady motion leaves it loose.
  • Leave Tg\mathbf{T}_g pinned unless the motion has strong linear acceleration while it rotates. Hand-held motion rarely does.

More: Observability and Degeneracy.