Calibration graph
Use this page to see everything MVIS solves at once, and in what order it gets there.
A factor graph links the unknowns (variables) to the measurements and priors that constrain them (factors). Each factor adds one term to the cost; the solver moves all variables together until the total is smallest. That joint solve is what lets a camera’s time offset, an IMU’s scale and a lever arm be fitted from one recording.
Symbols follow Notation & glossary. The factor equations are on Camera–IMU calibration math, Base IMU factor and Auxiliary IMU factor.
The camera–IMU graph
Section titled “The camera–IMU graph”The camera–IMU graph for one camera, one auxiliary IMU and five states. Each state holds the base-IMU pose, velocity and biases. Not drawn, to keep it readable: the reprojection factors also touch the anchor’s velocity and biases, the IMU intrinsics and t_ga; the auxiliary IMU has its own velocity and bias at each of its knots. The IMU calibration variables have priors; camera calibration gets one only on what is held (see Priors below).
Variables
Section titled “Variables”| Variable | Symbol | Size | How many |
|---|---|---|---|
| IMU pose | 6 | one per state, at solve.state_hz (30 Hz) on the first camera’s frames | |
| IMU velocity and biases | , , | 9 | one per state |
| Board pose | 6 | one; the board stays still | |
| Board shape | , one per board corner (Notation’s ) | 1 per corner | only with estimate.board_points: true |
| Camera calibration | , , , , | 8 + intrinsics | one per camera |
| Base IMU intrinsics | , , (and a triad rotation) | 24 | one |
| Gyro-to-accel offset | 1 | one per IMU, when estimate_t_ga is on (the default) | |
| Auxiliary IMU calibration | , , its intrinsics | 31 | one per auxiliary IMU |
| Auxiliary IMU velocity and biases | , , | 9 (3 if gyro-only) | one per auxiliary knot |
stacks all of them. A 60 s recording at 30 Hz has 1800 states, each with a pose, a velocity and two biases.
Factors
Section titled “Factors”| Factor | Links | Weight |
|---|---|---|
| Reprojection | one frame’s corners to its anchor state, the board, the camera calibration, the IMU intrinsics and t_ga | 1 px per axis; Huber past 2 on the frame’s stacked residual |
| Base IMU | two consecutive states, the IMU intrinsics, t_ga | the integration covariance |
| Bias random walk | the bias rows of the base and auxiliary IMU factors | the random-walk densities |
| Auxiliary IMU | two base states 3 intervals apart, the auxiliary states and calibration | the integration covariance |
| Priors | single variables (below) | per prior |
Rates. Each camera adds about 10 reprojection factors per second
(solve.factor_hz, settable per camera). The state rate stays 30 Hz, so the
trajectory stays dense even when the cameras are thinned.
Priors:
- Gauge on the first pose: a gauge prior fixes directions the data cannot see. Here it holds the first pose’s position and its rotation about gravity. The tilt (the direction of gravity in the IMU frame) stays free, since the accelerometer sees gravity.
- First velocity and biases: weak (0.5 m/s, 0.01 rad/s, 0.1 m/s²).
- IMU intrinsics: a tight prior on the pinned blocks, a weak one on the freed blocks (which blocks).
t_ga: zero-mean, 10 ms.- Camera calibration: free, except a tight prior on whatever is held:
on a global-shutter camera, and any
estimate.camera.*: false. - Board shape: each height has a 3 mm prior. A gauge prior holds the mean height and the two tilts at 0, so the board pose keeps the plane and the heights keep only the shape.
The cost
Section titled “The cost”The solve minimises the weighted sum of all factor errors, given on Camera–IMU calibration math. After the solve, MVIS computes the marginal covariance of every calibration variable. Its square root is the 1σ (one standard deviation) printed next to each value. A wide 1σ is how a weakly observed value shows itself.
Stages of a camera–IMU run
Section titled “Stages of a camera–IMU run”mvis cam-imu reaches the joint solve in steps. Each step gives the next a
starting point close enough to converge.
| Stage | What happens | What is estimated |
|---|---|---|
| 1. Window | Bags longer than 80 s: keep the 80 s with the most gyro and accelerometer excitation (input.window_mode: motion). Shorter bags: from 3 s in (input.t0). | — |
| 2. Detection | Board corners in every frame. | — |
| 3. Frame selection | Keep frames with enough corners and a valid board pose (PnP: the camera pose from known board points); thin each camera to factor_hz. | the board pose per frame, from the camera prior |
| 4. Start values | Per camera: time offset from angular rates, rotation and gyro bias from a rotation fit, then a linear solve for gravity, scale, velocities and lever arm; the accelerometer bias starts at 0. Per auxiliary IMU: offset, rotation and gyro bias from gyro-to-gyro. | rough , , , gyro bias, gravity direction; , |
| 5. Joint solve | The graph above, Levenberg–Marquardt, up to 50 iterations. t_ga re-integration as needed. | everything at once |
| 6. Clock retiming (opt-in) | For cameras marked retime_sensor_clock: a first solve gives each frame its own capture offset, a regular sensor clock is fitted to them, the frames are re-stamped, and the joint solve runs again. | per-frame offsets, then everything |
| 7. Results | Residuals; χ² per degree of freedom per factor type (the weighted squared error divided by its degrees of freedom, about 1 when the noise model fits); 1σ for every value. | — |
Checks on the way:
- Start values: a lever-arm start with 1σ over 5 cm starts at 0 instead. The start reports its gravity magnitude and scale (both should come out near 9.81 m/s² and 1) and how sharp the time-offset minimum is.
- The joint solve: Levenberg–Marquardt can stop at its damping cap far from a minimum, which looks like convergence. MVIS spots this from the step the linear system still predicts, solves again from the same start with a different damping, and keeps that result only if its cost is lower. A solve whose linear system cannot be solved (no step at all) is marked not ok.
Stages of a camera-only run
Section titled “Stages of a camera-only run”mvis cam has no IMU. Its graph holds one board pose per timestamp, the
intrinsics of every camera, the extrinsic of every camera to the first one, the
rolling-shutter readout, the board shape when asked, and time offsets between
cameras when optimize_time_offset is on (off by default).
| Stage | What happens | What is estimated |
|---|---|---|
| 1. Detection | Board corners in every frame. | — |
| 2. Start values | Per camera: OpenCV intrinsics (radtan) or a vanishing-point start (other models), and a board pose per frame. | intrinsics, board poses |
| 3. Gating | Drop frames with too few tags (6) or too small a spread, poses with a PnP error over 10 px, corners seen in too few frames. | — |
| 4. Per-camera solve | One graph per camera. | intrinsics, distortion, board poses, readout of rolling-shutter cameras |
| 5. Alignment | Align the board-pose tracks of the cameras. | camera-to-camera extrinsics, time offsets |
| 6. Joint solve | All cameras in one graph. | everything at once |
| 7. Outlier rerun | Drop corners and frames whose error is over 2 px and above the 99th percentile; solve again from the previous values. On by default. | everything again |
The camera-only solves after the start values thin the corners to two per tag
(corner_subsample: two_anti); the start values use every corner. The camera–IMU solve keeps all of them.
What this means for your recording
Section titled “What this means for your recording”- Record about 60 s. Past 80 s, MVIS keeps the 80 s with the most motion.
- Read the 1σ, not only the residual. A small residual with a wide 1σ means the value was not observed: see Degeneracy.
- Leave
solve.state_hzat 30 Hz. Lower rates undersample fast motion (step 4).