Digital-twin check
Step 7 of the walkthrough. Here you test the solver on a simulated copy of your recording, where the truth is known.
- You need: the camera–IMU config and bag from step 4, and MVIS installed from source or from the package.
- You get: the error of each calibration parameter against the truth.
- Check: the noise-free errors are near zero (section 2).
A digital twin is a simulated copy of your recording:
- The truth is your real solve: calibration, board pose and shape, biases and trajectory. A small
seeded perturbation then moves the camera intrinsics, the camera and aux-IMU extrinsics, each
tdand eacht_ga, by up to 1 % focal, 4 px principal point, 5 % of each distortion coefficient, 10 mrad per rotation axis, 5 mm per lever axis, 1 mstdand 0.5 mst_ga. Readout, IMU intrinsics, biases, gravity, board and trajectory stay as solved. The perturbation makes the solve recover values it was not given, and keeps your real calibration out of a twin you save or share.perturb_seed=Noneincapturekeeps the real solve as it is. - The timing is real: MVIS keeps the real frame times, the corner ids seen in each frame and the IMU stamps.
- The data is new: MVIS regenerates the corners and the IMU readings from the truth, with rolling shutter and t_ga (the time offset between the gyroscope and the accelerometer).
- The solve is the real pipeline, run again on the new data.
So any gap between the result and the truth comes from the solver, not from the recording.
1. Build and solve a twin
Section titled “1. Build and solve a twin”The twin is the Python module mvis.simulation.twin.
from mvis.simulation import twin
# Runs the real solve once (it needs the bag) and takes it as the truth.# noise_from_residuals sets the noise to match the real residuals.tw = twin.capture("virig_4imus_4cams_cam_imu.yaml", "virig4", noise_from_residuals=True)
report = twin.solve(tw, seed=0) # with noiseclean = twin.solve(tw, noise=False) # without noise
tw.save("virig4_twin.npz") # keep it; it holds no imagestw = twin.Twin.load("virig4_twin.npz")captureruns your camera–IMU config once on the real bag. Its outputs go toresults/twin_capture(set another folder withwork_dir=).noise_from_residuals=Truemeasures the corner noise from the real solve. It runs one extra solve of the twin to scale the IMU noise, so the twin’s IMU fit matches the real one.solvesimulates, solves and returns the errors (estimate minus truth).seedpicks the noise draw. Run a few seeds, not one.- A saved twin needs no bag. Solve it again after any change to MVIS or to the solve settings.
2. Read the errors
Section titled “2. Read the errors”Each solve returns, per sensor:
- Cameras: lever arm (mm), rotation as a log(R) vector (mrad), td (the camera–IMU time offset, ms), readout (ms), focal length (px).
- Base IMU: t_ga (ms), gyroscope and accelerometer scale (%).
- Each aux IMU: lever, rotation, td, t_ga and scales.
- Board: rms height error (mm), when the board shape is estimated.
Read the two kinds of run side by side:
- The noise-free error should be near zero. If it is not, the solver does not recover the truth even from perfect data. That is a model or observability problem.
- The error with noise shows what your data can support. Large errors with noise and small noise-free errors mean the data limits the result, not the solver.
3. Watch the twin of the full rig
Section titled “3. Watch the twin of the full rig”We ran the step-4 config (all 4 cameras, all 4 IMUs, bag 01) and simulated the same 80 s from its result. Each video shows:
- Left: the IMU trajectory and the board, with each camera’s pose.
- Right: each camera’s corners, observed (circles) and reprojected (dots), with the RMS for that frame.
- Bottom: position (m) and orientation as log(R) from the start (rad), with a cursor at the current time.
This video twin is simpler than the twin module of section 1:
- Truth: the real solve as it is, with no perturbation: trajectory, board, every camera and IMU.
- Cameras: all four at 30 Hz, each seeing every tag in its view, with 0.5 px corner noise.
- IMUs: 200 Hz, with a fixed noise model (gyroscope 1.7e-4 rad/s/√Hz, accelerometer 3e-3 m/s²/√Hz).
So its corners are cleaner than the real ones. On real data the per-frame RMS jumps when the board is blurred or far away.
Solved against its truth (one noise draw, all 4 cameras and all 4 IMUs, reprojection RMS 0.71 px):
| Sensor | Lever error | Rotation error | Readout error |
|---|---|---|---|
| T265 left | 0.13 mm | 0.17 mrad | — |
| T265 right | 0.14 mm | 0.14 mrad | — |
| ELP-L | 0.16 mm | 0.12 mrad | 0.05 ms |
| ELP-R | 0.14 mm | 0.16 mrad | 0.04 ms |
| GX3-25 (to the base IMU) | 0.18 mm | 0.54 mrad | — |
| GX3-35 (to the base IMU) | 0.24 mm | 0.23 mrad | — |
| T265 IMU (to the base IMU) | 0.18 mm | 0.45 mrad | — |
Every lever lands within 0.25 mm and every rotation within 0.6 mrad of the truth. The solve converges on the full rig.
4. Twin of the ELP pair
Section titled “4. Twin of the ELP pair”We built a twin module twin of the ELP-pair camera–IMU run on
virig-4imus-4cams bag 01:
- Cameras: ELP left and right, rolling shutter.
- IMUs: the T265 IMU as base, the Xsens as aux IMU.
- Settings: as in the “ELP RS stereo + two IMUs” run on the results page: camera intrinsics free, t_ga on, board shape on.
Errors against the truth (estimate minus truth). For lever and rotation the table gives the length of the 3D error; for a scale, the largest of its three axis errors.
| Parameter | Noise-free | With noise, RMS over seeds 0–2 |
|---|---|---|
| Camera lever | 0.094 mm | 0.44 mm |
| Camera rotation | 0.177 mrad | 0.32 mrad |
| Camera td | 0.007 ms | 0.024 ms |
| Camera readout | 0.003 ms | 0.017 ms |
| Camera focal length | 0.009 px | 0.096 px |
| Base t_ga | 0.015 ms | 0.053 ms |
| Base gyroscope scale | 0.011 % | 0.108 % |
| Base accelerometer scale | 0.013 % | 0.033 % |
| Board height rms | 0.23 mm | 0.23 mm |
| Aux lever | 0.164 mm | 0.20 mm |
| Aux rotation | 0.245 mrad | 0.58 mrad |
| Aux td | 0.007 ms | 0.036 ms |
The camera rows are ELP left; ELP right is close. What it shows:
- The solver converges to the truth. Without noise every error is a fraction of a mm, mrad or ms, and the scales land within 0.02 %. This is the check that matters.
- Noise sets the limit, and it matches real data. With noise, the gyroscope scale lands within 0.11 %, t_ga within 0.05 ms and the ELP readout within 0.02 ms. That is no larger than the real spread over the 15 recordings, which also vary the motion: T265 gyroscope scale sd 0.03–0.13 % per axis, t_ga sd 0.13 ms, ELP readout sd 0.12 ms (results page, “ELP RS stereo + two IMUs”).
- The board height keeps about 0.2 mm of error even without noise.
A check for every MVIS change. We keep these errors as a reference and re-run the twin for new MVIS versions. A version fails if a solve fails, takes no step, or an error grows past the reference by more than:
| Quantity | Allowed growth (noise-free) |
|---|---|
| Lever | 0.2 mm |
| Rotation | 0.2 mrad |
| td, readout | 0.05 ms |
| t_ga | 0.1 ms |
| Gyroscope scale | 0.01 % |
| Accelerometer scale | 0.02 % |
| Focal length | 0.2 px |
| Board height rms | 0.1 mm |
Errors with noise may grow by twice these amounts, as three seeds are a rough sample. Smaller errors never fail.
5. Twin of the motion
Section titled “5. Twin of the motion”We also used twins to choose the recording motion of step 2. A simulated Looper Insight 9 ran each motion on 3 noise seeds and once without noise. See how we judged it.
Without ground truth
Section titled “Without ground truth”On real data there is no truth. Two checks stand in for it, strongest first:
- Round trip:
mvis roundtripapplies the calibration to its own bag and calibrates again. PASS needs the IMU intrinsics to collapse and the geometry to hold. - Repeatability: calibrate several recordings of the same rig with batch mode. A parameter that moves between recordings was not pinned down by their motion.
Simulate without a recording
Section titled “Simulate without a recording”To try a rig or a motion you have not recorded, start from a simulator example. Each one holds a real motion and a full rig setup (cameras, IMUs, board), with a perturbed truth like a twin:
from mvis.simulation.examples import load_example, simulatefrom mvis.calibration.pipeline_cam_imu import run_cam_imu_calibration_multi
ex = load_example("twin_gs") # ships with the packagedata = simulate(ex, seed=0) # cams, t_imu, w_m, a_m, aux_imusresult = run_cam_imu_calibration_multi( data["cams"], ex.board_points_flat, data["t_imu"], data["w_m"], data["a_m"], aux_imus=data["aux_imus"] or None)- In the package:
twin_gs, a global-shutter stereo twin with one IMU. - In a source checkout (
data/sim_examples/): more examples, among themvirig_4imus_3cams(our VI-Rig),elp_t265,tum_vi_raw,twin_rs(a rolling-shutter twin),guide06c_60(the AprilGrid guide motion of step 2) andguide60b(an earlier chessboard motion).mvis.simulation.examples.available()lists what your install has. - Licences: each example’s
setup.yamlnames its licence; check it before you reuse a guide motion. - Lower level:
mvis.simulation.scenariobuilds a dataset from a trajectory, a true calibration and a board placed in the world.
For the theory of weak motions, see Degeneracy.
Step 8: round-trip validation checks the calibration on real data.