Skip to content

Rolling-shutter calibration

Step 5 of the walkthrough. Here you solve the readout time of the two rolling-shutter ELP cameras.

  • You need: the camera-only config from step 3 and the camera–IMU config from step 4.
  • You get: a readout time for each ELP camera.
  • Check: the readout repeats across recordings (section 3).

Two terms:

  • Rolling shutter: the camera exposes its rows one after another, so one image spans a range of times.
  • Readout: the time from the first row to the last row of one image.

In the running example the two T265 fisheyes are global shutter (all rows at once) and the two ELP cameras are rolling shutter. One rig can mix both.

1. Camera-only: mark the ELP as rolling shutter

Section titled “1. Camera-only: mark the ELP as rolling shutter”

The camera-only ELP config of step 3 already sets shutter_type: rolling on each ELP camera. That one key turns on the readout estimate:

elp_cam.yaml (excerpt)
input:
cameras:
- camera_id: 0
name: ELP-L
# ... other keys as in step 3
shutter_type: rolling # global | rolling | auto (auto = global)
- camera_id: 1
name: ELP-R
# ... other keys as in step 3
shutter_type: rolling
shell
mvis cam -c elp_cam.yaml
  • What it does: mvis cam solves the readout of every camera marked rolling.
  • Global cameras: leave the T265 fisheyes at shutter_type: global.
  • auto: the default. It counts as global, so no readout is solved.

Result over the 15 recordings (results page, “ELP RS stereo + readout”, camera only), median and standard deviation (sd):

Readout (ms)ELP-LELP-R
MVIS14.604 (sd 0.404)14.601 (sd 0.404)
Kalibr, rolling-shutter model16.354 (sd 2.00)17.369 (sd 1.42, n = 13)
  • Kalibr: its rolling-shutter model calibrates one camera per run. Two ELP-R recordings failed, so it has 13 of 15.
  • Lens models differ: MVIS uses equidistant, Kalibr radtan. So only the readout is compared.
  • Kalibr inputs: Kalibr was not re-run after the ELP stamp correction, so its rows come from the bags as captured. It also assumes a flat board; MVIS estimates the board shape. Do not read the whole 2–3 ms gap as a tool difference.

2. Camera–IMU: turn readout on per camera

Section titled “2. Camera–IMU: turn readout on per camera”

In the step 4 config, the camera_input.cameras list holds one entry per camera:

virig_4imus_4cams_cam_imu.yaml
camera_input:
cameras:
- {}
- factor_hz: 4.0
- estimate_readout: true
factor_hz: 4.0
- estimate_readout: true
factor_hz: 4.0
estimate:
camera:
tr: false
  • Order: the list follows the camera order in input.cam_topics. cam0 and cam1 are the T265 fisheyes, cam2 and cam3 the ELP left and right. (In the camera-only elp_cam.yaml above, the ELP cameras are camera_id 0 and 1.)
  • estimate_readout: true: solves the readout of that camera.
  • factor_hz: 4.0: adds that camera’s frames to the solve at 4 Hz, which keeps the solve small.
  • estimate.camera.tr: false: the rig-wide default, which keeps readout off for the global-shutter T265. The per-camera estimate_readout: true wins for the ELP.
  • Readout on every camera: set estimate.camera.tr: true and leave out the per-camera key.
shell
mvis cam-imu -c virig_4imus_4cams_cam_imu.yaml

The full config is on the example page.

Recording 01 (example run):

ELP-L (cam2)ELP-R (cam3)
Readout (ms)16.68416.732
td (ms)−123.25−123.30
  • td: the camera’s time offset to the base IMU. The ELP td is large; the T265 cameras sit at −5.41 and −7.47 ms.

All 15 recordings (results page, MVIS, median and sd):

Run on the results pageELP-L readout (ms)ELP-R readout (ms)
ELP RS stereo + two IMUs16.618 (sd 0.122)16.594 (sd 0.111)
Mixed t265-f1 + ELP-L, aux Xsens16.680 (sd 0.089, n = 14)—
  • Repeatable: the camera–IMU readout moves by about 0.1 ms between recordings. A readout that moves much more was not observed by the motion.
  • Check it on your rig: calibrate several recordings from one config with batch mode and compare.

4. Camera-only and camera–IMU readouts differ

Section titled “4. Camera-only and camera–IMU readouts differ”
  • The gap: on these recordings camera-only gives 14.6 ms, camera–IMU 16.6 ms. We have not found the cause yet.
  • Which to use: the camera–IMU value when you have an IMU. It repeats about 4 × more tightly (sd about 0.1 ms against 0.4 ms).

The readout is fitted by the same residuals as everything else, not applied as a fix afterwards.

  • Per-row time: each corner is projected with the rig pose at the time of its row, not of the frame.
  • One error term: projection, td, readout and pose interpolation sit in one factor.

The math: Camera model.

The readout is one of the weakest parameters in the solve.

  • It needs fast motion: the rows must see clearly different geometry.
  • A slow, smooth recording will not pin it. The estimate then stays near where it started.
  • Repeatability: see section 3.
  • The round trip cannot test it: mvis roundtrip corrects the bag with mvis apply-calib, which re-stamps each frame but does not unwarp its rows. So a readout error stays hidden.
  • The digital twin can: it simulates the rows. See step 7.