Camera model
Use this page to pick the camera model for each camera in your rig, and to see the exact equations MVIS uses.
A camera model maps a 3D point in the camera frame to a pixel : a projection plus a lens distortion. Symbols follow Notation & glossary.
Each model lives in one class under
src/mvis/camera/.
The class holds the projection, its Jacobians (the derivatives the optimiser
uses) and the unprojection. So every model works:
- in camera-only and camera–IMU calibration;
- with a calibration board of known corner positions, and in feature mode, where the corners’ 3D positions are not given and are solved as well (example: VI-Rig 4 IMUs + 3 cams, ArUco);
- mixed across cameras in one rig.
The eight models
Section titled “The eight models”| Model | Config value | Origin | Distortion coefficients, in file order | Suits |
|---|---|---|---|---|
| Pinhole + radial-tangential | radtan | Brown–Conrady | k1 k2 p1 p2 | Normal lenses |
| Kannala–Brandt KB4 | kb4 (also fisheye, kannala_brandt) | Kannala–Brandt, from atan2 | k1 k2 k3 k4 | Fisheye, including corners past 90° |
| Equidistant | equidistant | Kannala–Brandt on | k1 k2 k3 k4 | Fisheye with every corner under 90° |
| FOV | fov | Devernay–Faugeras | omega | Wide-angle, one parameter |
| Double Sphere | double_sphere | Usenko et al. (Basalt) | xi alpha | Very wide field of view |
| EUCM | eucm | Khomutenko et al. | alpha beta | Very wide field of view |
| UCM | ucm | Unified model (Mei), form | alpha | Very wide field of view; Kalibr omni / none |
| Omni + radtan | omni_radtan | Mei, experimental | xi k1 k2 p1 p2 | Catadioptric, ultra-wide |
Every model shares the four intrinsics (fx fy cx cy).
The distortion coefficients follow them.
cameras: - camera_id: 0 distortion_model: kb4Projection equations
Section titled “Projection equations”Each model ends with the same affine step from a normalised point to pixels:
What differs is how comes from . A point outside a model’s valid domain has no pixel. That corner is left out of the error statistics and, in camera–IMU, out of the cost. Left uncounted, it would make the error look smaller than it is. So the result counts dropped corners:
past_fold: a Kannala–Brandt corner past the fold;other_invalid: anything else, such as underequidistant.
radtan
Section titled “radtan”Source: src/mvis/camera/pinhole_radtan.py (GTSAM Cal3DS2).
Valid: .
kb4 and equidistant
Section titled “kb4 and equidistant”Both use the Kannala–Brandt odd polynomial in the incidence angle (the angle of the ray off the optical axis):
The two differ only in how they get :
| Model | Valid domain | |
|---|---|---|
kb4 | and below the first fold | |
equidistant | , after dividing by | , so |
The first fold is the smallest where ( when there is none). Past it, falls again and two rays share one pixel, so the inverse is no longer unique.
- Below 90° the two models give the same pixels for the same coefficients.
- At or past 90° only
kb4has a pixel. This is whyfisheyemeanskb4(Why kb4 for fisheye). - After a solve MVIS logs the fold angle and the radius in pixels where the valid domain ends. It warns when that radius lies inside the image.
Source: src/mvis/camera/kb4.py,
src/mvis/camera/kb_domain.py,
src/mvis/camera/pinhole_equidistant.py (GTSAM Cal3Fisheye).
Source: src/mvis/camera/fov_model.py.
On the optical axis the scale is its limit . As the model becomes a plain pinhole. Valid: .
double_sphere
Section titled “double_sphere”Source: src/mvis/camera/double_sphere.py.
Valid: . Expected ranges: , .
Source: src/mvis/camera/eucm.py.
Valid: . Expected ranges: , .
Source: src/mvis/camera/ucm.py.
UCM is EUCM with :
Valid: , with for , else . Past that bound the image folds back. Expected range: .
Kalibr’s omni camera with distortion none is the same model in form.
MVIS converts on import and export only:
omni_radtan (experimental)
Section titled “omni_radtan (experimental)”Source: src/mvis/camera/omni_radtan.py.
A projection onto the unit sphere with mirror parameter , then radtan
distortion on the normalised plane:
then from exactly as in radtan.
Valid: . Expected range: .
This is Kalibr’s omni / radtan.
All projection kernels outside GTSAM are in
src/mvis/calibration/factors/custom_projections.py,
with analytical Jacobians.
Choosing one
Section titled “Choosing one”More parameters are not better. A model richer than the lens needs fits noise. On wide field-of-view data it can slide into a basin (a wrong local minimum) that looks converged and is not.
Choose by the largest incidence angle of the board corners, not by the lens’s
advertised field of view. Every result reports this angle
(max_theta_deg, and max_theta_used_deg for the corners the fit kept), and
counts any corners past the model’s valid domain.
- Normal lens:
radtan. - Fisheye:
kb4, which is whatfisheyeandkannala_brandtselect. It stays valid past 90° up to its first fold (why). - Every corner under 90°, and you want equidistant by name:
equidistant. It gives the same pixels askb4there. - Very wide field of view, a different shape:
eucm,ucmordouble_sphere. - Catadioptric (lens plus mirror):
omni_radtan. It is experimental.
Kalibr export
Section titled “Kalibr export”The camchain written for Kalibr uses these names:
| MVIS | Kalibr camera_model / distortion_model |
|---|---|
radtan | pinhole / radtan |
equidistant | pinhole / equidistant |
kb4 | pinhole / equidistant: exact for ; MVIS warns when the fit used corners at or past 90°, which Kalibr cannot project |
fov | pinhole / fov |
double_sphere | ds / none |
eucm | eucm / none |
ucm | omni / none, converted to form |
omni_radtan | omni / radtan |
Camera-only calibration runs two passes: a first solve, then a second one
after outlier rejection. A guardrail keeps the first-pass result if the
second pass lands on any of these
(src/mvis/calibration/pipeline_cam.py):
- a focal length of 1 px or less;
- and more than 30 % apart;
- a distortion coefficient above 2 in magnitude (for
ucm: outside ; foromni_radtan: outside ); - a 95th-percentile reprojection error more than 1.5 times the first pass’s, on a camera whose moved less than 5 %;
- a mean RMS reprojection error over all cameras more than 1.5 times the first pass’s.
It is a backstop, not a substitute for picking the right model.
Rolling shutter
Section titled “Rolling shutter”A rolling-shutter camera exposes its rows one after another, so one image spans a range of times. The readout is the time from the first row to the last.
A corner observed in row of an image of height was captured at
The row factor depends on which row the driver stamps
(rolling_mode, default 0):
rolling_mode | Frame stamp at | |
|---|---|---|
| 0 | top row | |
| 1 | middle row | |
| 2 | bottom row |
Source: src/mvis/core/shutter.py.
For a global-shutter camera , so every corner of a frame shares one time.
The camera–IMU vision factor uses that per-corner time directly. It takes the pose of at from the base IMU, composes the extrinsics, and then projects:
where is board corner in the camera frame at the corner’s capture time. Projection, time offset, readout and extrinsics sit in one factor, so and get Jacobians and are solved with everything else, not corrected afterwards. The HTML report uses the same projection code, so the residuals it shows are the ones the optimiser minimised.
Turn it on per camera:
cameras: - camera_id: 0 shutter_type: rollingOr through the estimate: block, with estimate.camera.tr: true.
Practical guidance: Rolling shutter.