Skip to content

Observability

Use this page to see which motion makes which calibration value measurable, and why.

A value is observable when the measurements change if the value changes. If they do not, the data cannot tell a right value from a wrong one, however long you record. The solver then returns roughly its start value, with a wide 1σ.

Whether a calibration value is observable depends on the motion. The MVIS paper analyses this for the full multi-camera, multi-IMU state, and the earlier papers below give the degenerate motions for the time offset, extrinsics and IMU intrinsics:

This page gives the short reason for each value. Symbols follow Notation & glossary; the factors are on Camera–IMU calibration math.

The measurements are relative: the cameras see the board, the IMUs feel motion and gravity. Moving the whole scene (board and trajectory) or turning it about gravity changes nothing they measure. So four directions are never observable:

  • global position (3);
  • rotation about gravity (1).

MVIS holds them with a prior on the first state’s position and its rotation about gravity. The tilt (the direction of gravity in the IMU frame) stays free: the accelerometer sees gravity, so it is observable. Metric scale is observable too: the board has a known size, and the accelerometer measures in m/s².

ValueNeedsShort reason
Camera intrinsics fi\mathbf{f}_i, di\mathbf{d}_ithe board seen at many distances, angles and across the whole imagefrom the board alone
Extrinsic rotation CiIR{}^{I}_{C_i}\mathbf{R}rotation about at least 2 axescamera and gyro see the same angular velocity, rotated
Lever arm IpCi{}^{I}\mathbf{p}_{C_i}rotation about at least 2 axes, with changing angular velocityit shows up only through rotation
Time offset td,it_{d,i}angular velocity that changes, and some translationa time shift moves a pose by its velocity
Readout tr,it_{r,i}fast rotation about the camera’s x or y axis, board spread over many rowseach row is shifted by its own share of the readout
Gyro scale and misalignment Dw\mathbf{D}_wrotation about every axis, at varied rateseach column multiplies one axis of the angular velocity
Accel scale and misalignment Da\mathbf{D}_a, bias ba\mathbf{b}_agravity along every axis (tilts), plus linear acceleration changeseach column multiplies one axis of the specific force
G-sensitivity Tg\mathbf{T}_gstrong linear acceleration while rotating on all axesthe gyro error must vary with acceleration, apart from rotation
t_gaquickly changing acceleration (brisk reversals)it enters only through the rate of change of acceleration
Auxiliary IMU rotation and td,jt_{d,j}rotation about at least 2 axesthe two gyros see the same angular velocity
Auxiliary IMU lever armrotation about at least 2 axes, with changing angular velocitycentripetal and tangential acceleration

The motion in step 2: record the dataset is built to cover every row of this table.

The camera’s angular velocity (from its board poses) and the gyro’s are the same vector in two frames:

Iω(t)=CiIR  Ciω(t)(after time alignment){}^{I}\boldsymbol\omega(t) = {}^{I}_{C_i}\mathbf{R}\;{}^{C_i}\boldsymbol\omega(t) \quad\text{(after time alignment)}

A rotation is fixed by two non-parallel vector pairs. If every Ciω{}^{C_i}\boldsymbol\omega is parallel to one axis, any rotation about that axis fits equally well. The start-value step of MVIS records the singular values of the camera–gyro angular-velocity cross-covariance for this reason: one large and two tiny ones mean one-axis rotation.

The camera sits at GpC=GpI+IGR IpC{}^{G}\mathbf{p}_{C} = {}^{G}\mathbf{p}_{I} + {}^{G}_{I}\mathbf{R}\,{}^{I}\mathbf{p}_{C}. Differentiate twice and compare with what the accelerometer measures:

Gp¨C=Gp¨I+IGR([Iω˙]×+[Iω]×2) IpC{}^{G}\ddot{\mathbf{p}}_{C} = {}^{G}\ddot{\mathbf{p}}_{I} + {}^{G}_{I}\mathbf{R}\Big(\big[{}^{I}\dot{\boldsymbol\omega}\big]_\times + \big[{}^{I}\boldsymbol\omega\big]_\times^2\Big)\,{}^{I}\mathbf{p}_{C}
  • No rotation: both terms vanish and IpC{}^{I}\mathbf{p}_{C} is a constant offset the board pose absorbs.
  • Rotation about one axis k\mathbf{k}: a component of IpC{}^{I}\mathbf{p}_{C} along k\mathbf{k} gives k×p=0\mathbf{k}\times\mathbf{p} = 0, so that component stays unobservable.

The same holds for the lever arm of an auxiliary IMU.

Time offset: a shift is seen through velocity

Section titled “Time offset: a shift is seen through velocity”

A time offset error δt\delta t moves the pose used for a frame to a slightly different time:

IGT(t+δt)≈IGT(t)  Exp ⁣([IωIv]δt){}^{G}_{I}\mathbf{T}(t + \delta t) \approx {}^{G}_{I}\mathbf{T}(t)\;\mathrm{Exp}\!\left(\begin{bmatrix}{}^{I}\boldsymbol\omega\\ {}^{I}\mathbf{v}\end{bmatrix}\delta t\right)
  • At rest the shift changes nothing.
  • At constant angular and linear velocity the shift is a constant relative transform. The extrinsic absorbs it, and td,it_{d,i} and the extrinsic cannot be told apart.
  • When the velocity changes the shift varies over the recording and no constant transform can explain it. That is the signal.

On a rolling-shutter camera, a corner on row vmv_m is captured at tC+td,i+(vm/H) tr,it_C + t_{d,i} + (v_m/H)\,t_{r,i}. The readout is a time offset that grows from the top row to the bottom row. It needs motion that moves the rows differently: rotation about the camera’s x or y axis, which sweeps the image vertically or horizontally. Rotation about the optical axis gives little signal, and so does a board that covers only a few rows: then tr,it_{r,i} cannot be told from td,it_{d,i}. See Why calibrate rolling shutter.

IMU scale and misalignment: every axis excited

Section titled “IMU scale and misalignment: every axis excited”

The corrected readings are Iω=wIR Dw(ωm−bg−Tg Ia){}^{I}\boldsymbol\omega = {}^{I}_{w}\mathbf{R}\,\mathbf{D}_w(\boldsymbol\omega_m - \mathbf{b}_g - \mathbf{T}_g\,{}^{I}\mathbf{a}) and Ia=aIR Da(am−ba){}^{I}\mathbf{a} = {}^{I}_{a}\mathbf{R}\,\mathbf{D}_a(\mathbf{a}_m - \mathbf{b}_a) (Base IMU factor). Column nn of Dw\mathbf{D}_w multiplies only axis nn of the reading. If that axis never turns, the column has nothing to scale.

The accelerometer measures specific force, Ia=GIR (Gp¨I−Gg){}^{I}\mathbf{a} = {}^{I}_{G}\mathbf{R}\,({}^{G}\ddot{\mathbf{p}}_I - {}^{G}\mathbf{g}). Gravity is about 9.8 m/s², far larger than hand-held acceleration. So tilting the rig until each axis has pointed along gravity is what excites Da\mathbf{D}_a. A rig held at one tilt sees one fixed gravity vector: then the scale and the bias along it trade off against each other.

G-sensitivity: acceleration must vary apart from rotation

Section titled “G-sensitivity: acceleration must vary apart from rotation”

Tg Ia\mathbf{T}_g\,{}^{I}\mathbf{a} adds to the gyro reading. With mostly gravity in Ia{}^{I}\mathbf{a}, its effect looks like a gyro bias that changes with tilt, which the biases and Dw\mathbf{D}_w can also explain. It separates only with strong linear acceleration on all axes. Hand-held motion rarely gives that, so it stays pinned by default.

t_ga: seen through the change in acceleration

Section titled “t_ga: seen through the change in acceleration”

The accelerometer reading that belongs to gyro time tt is the one stamped t+tgat + t_{ga}. To first order:

am(t+tga)≈am(t)+tga a˙m(t)\mathbf{a}_m(t + t_{ga}) \approx \mathbf{a}_m(t) + t_{ga}\,\dot{\mathbf{a}}_m(t)

The offset is visible only through a˙m\dot{\mathbf{a}}_m, the rate of change of the specific force. Smooth, steady motion leaves it loose; brisk reversals make it sharp. The response of some IMUs also depends on the frequency of the motion, so one value can be a compromise: see Why calibrate t_ga.

Observable is not yes or no in practice. A value can be observable in theory and still weak on a given recording: the 1σ then comes out wide, and the value moves from one recording to the next. Two ways to check:

  • Read the 1σ next to each value in the result. Compare it across recordings.
  • Run the digital twin of your own rig and motion: MVIS simulates the recording with known truth and solves it (step 7).

What happens when a motion is missing, and how MVIS reports it: Degeneracy.

  • Rotate about all three axes, at varied speed, with the board in view.
  • Tilt the rig through many orientations, so gravity falls along every axis.
  • Change direction briskly a few times. Avoid long stretches of steady motion.
  • Sweep the board across all image rows, more so for a rolling-shutter camera.
  • Hold still for about 3 s at both ends. The rests give the motion a clean start and end; the start-value step can compare a still start with its gyro-bias estimate.