Why kb4 for fisheye

Use this page to see why kb4 is the MVIS fisheye model and equidistant is not. You need nothing to run here; the numbers come from our runs.

The answer first:

One polynomial: both models share it. Below 90° off axis the formulas are identical; fitted separately on the T265, they agree to about 2e-6 px.
Past 90°, only kb4 can project: equidistant has no projection for rays at or past 90°; kb4 has one.
On the one past-90° lens we tested, equidistant stalled: on BF2M (up to 97° off axis) the equidistant solve stayed at its starting values, 15–24 px off TUM in fx, even on a recording where it dropped no corner. We have not found why. kb4, from the same start, came within 0.4 px of TUM in fx.
No cost below 90°: a camera–IMU result with kb4 exports to Kalibr as pinhole-equi (exact for rays in front of the camera; MVIS warns when the fit used corners past 90° or folds inside the image), and fisheye / kannala_brandt mean kb4. On a lens past 90°, the fitted polynomial can fold inside the image (BF2M: no ray past about 712–743 px from the centre).

How to set it: Choose a fisheye model (T265 example).

The two models

Both models map the angle θ of a ray off the optical axis to a distance r from the principal point, with the same four coefficients:

r = f · θ (1 + k₁θ² + k₂θ⁴ + k₃θ⁶ + k₄θ⁸)

They differ only in how they get θ from a 3D point (x, y, z):

Model	θ from	Rays at or past 90° (z ≤ 0)
`equidistant` (Kalibr `pinhole-equi`)	atan(√(x² + y²) / z), after dividing by z	Cannot be projected
`kb4` (Kannala–Brandt, as in Basalt)	atan2(√(x² + y²), z)	Projected, up to the polynomial’s first fold

1. Below 90°, they are the same

Where few or no corners pass 90°, the two models give the same calibration:

Data	Solves	kb4 − equidistant
Intel T265, VI-Rig, 7 recordings, camera-only	28 pairs	mapping gap p95 ≤ 1.9e−6 px, intrinsics ≤ 7.8e−7 px
Intel T265, VI-Rig, 7 recordings, camera–IMU	28 pairs	rotation ≤ 1.0e−11 rad, lever ≤ 1.9e−9 mm, `td` ≤ 1.5e−10 ms
TUM-VI, imu1–4, camera–IMU	8 pairs	rotation ≤ 2.2e−13 rad
TUM-Fisheye, 4 lenses (largest corner 58–89°, by TUM’s calibration), 12 recordings	24 pairs (one and all corners)	same fx to 0.01 px

So on the T265 class, regression rows that use equidistant stand for kb4 too. The T265 and TUM-VI rows are from MVIS main 644f7a2, the TUM-Fisheye row from 197549a. We have not re-run them on newer code.

2. On a lens past 90°, equidistant stalled

The TUM-Fisheye BF2M2020S23 lens sees the board up to 94–97° off axis (by MVIS’s kb4 fit). Each recording has 13–19 thousand detected corners (11.8–18.3 thousand in the kb4 solve). TUM’s own calibrations of the same images are the reference; they are another tool’s results, not ground truth.

Recording	Largest angle (kb4)	equidistant fx − TUM	kb4 fx − TUM	Mean error, equidistant / kb4
1	94.0°	+23.88 px	+0.06 px	1.35 / 0.16 px
2	97.0°	+15.51 px	+0.31 px	0.95 / 0.20 px
3	96.1°	+22.07 px	+0.33 px	1.48 / 0.20 px

All corners, camera-only, MVIS main f902d39 (2026-11-11). With two corners per tag, the MVIS default, the equidistant intrinsics are the same (mean error within 0.01 px) and kb4 is +0.12 to +0.39 px off TUM in fx.

What happens: both models start from the same guess: fx = fy = 404.98 / 396.51 / 402.95 px, the principal point at the image centre (639.5, 511.5) and zero distortion. kb4 moves from there. Equidistant does not: the solver gives up at the first step with the error unchanged, so every intrinsic ends at its starting value. The 15–24 px fx offsets are that starting guess.
Where it gives up: every trial step raises the cost (recording 1: 15,541 at the start, 85,047 down to 42,972 as the damping lambda rises from 1e-5 to 1e1), and the solver stops at MVIS’s damping cap (lambda 1e2; GTSAM’s default is 1e5). We have not tested whether a higher cap lets equidistant converge.
kb4’s other intrinsics: cx is 0.30–0.48 px low, cy within 0.1 px, and fy within 0.37 px of TUM.
The corners past 90° are not the whole story: equidistant’s stalled fit drops 0, 3 and 1 corners as past 90°. At its starting values, recording 1’s fit puts no corner past 90° (kb4’s fit puts its largest at 94°), yet it stalls the same way. Whether the corners near or past 90° block the first step is not tested. We have not found why the solve stalls.
Warnings: since PR #45 (commit a39f357, merged as 5d28769, 2026-11-05), MVIS warns when corners lie at or past 90° with equidistant and suggests kb4. That fires on recordings 2 and 3. Nothing in MVIS flags the stall: on recording 1 the only signs are the solver library’s “Levenberg-Marquardt giving up” line, an error that does not change (15541.4565 -> 15541.4565), and a mean error about 8× kb4’s.

Earlier runs, not repeated on f902d39:

Corners past 90°: by TUM’s calibration, 36, 37 and 45 corners lie past 90°, counted at MVIS 197549a on the frames of the kb4 all-corner run (the same corner totals as the kb4 solve above).
The old result was the same: at MVIS 197549a (2026-11-04) the three all-corner equidistant solves gave the same values to 4 decimals.
One corner per tag converged: at 197549a, with one corner per tag, equidistant did converge on the same bags (fx −0.00 / +0.91 / +2.72 px from TUM; kb4 −0.00 / +0.14 / +0.15), so the stall depends on the corner set. On the other four TUM-Fisheye lenses it converged with one and with all corners (section 1).
Kalibr did not do better: its pinhole-equi failed to converge on all 3 BF2M recordings, even seeded with TUM’s fx (it also failed on 1 of the 12 other recordings). Its DS and EUCM fits converged on all 3, after a patched initial guess on a second attempt.

3. Choosing kb4 costs nothing below 90°

Same pixels below 90° (section 1), so nothing changes on a lens that never gets there.
Kalibr export: a camera–IMU result with kb4 is written to Kalibr’s camchain-imucam as pinhole-equi, with the same k₁–k₄. It is exact for rays in front of the camera. MVIS warns when the fit used corners past 90°, or when the polynomial folds inside the image. Camera-only results have no Kalibr export.
The fold is checked: a KB polynomial can turn back (r stops growing with θ). MVIS reports the fold angle of every fit and flags corners past it. On TUM-VI (MVIS b03d456, all corners), 13 of 16 fits fold inside the image (12 of 16 with two corners per tag), but beyond every corner (smallest margin 22.2°): no corner lies past the fold, although MVIS warns that the fold is inside the image.
On BF2M the fold is inside the image too: every kb4 fit folds at 120–128°, 712–743 px from the principal point, while the image corner is 822 px away. No corner lies past the fold (the largest is at 97°), but pixels beyond that radius have no ray. MVIS warns about it in every run.

Conclusion: kb4 is the same model as equidistant where both work, and keeps working on the one past-90° lens we tested, where equidistant stalled. Since MVIS 29daf90, fisheye and kannala_brandt mean kb4. Use equidistant only when you want that name, on a lens whose corners stay under 90°.

Limits

Past-90° evidence is thin. It is one lens (BF2M, 3 recordings, 36–45 corners past 90° each) plus TUM-VI, where 96 corners past 90° are detected and about 40 reach the solve. Basalt detects 2,481 there; those TUM-VI counts are from a one-off survey with our own solver, not yet re-checked. The reference is TUM’s calibration, not ground truth.

Not tested: why equidistant stalls on BF2M (at its starting values, recording 1’s fit puts no corner past 90°); whether it converges from another start, such as TUM’s fx; and whether it converges with a higher damping cap (MVIS stops at lambda 1e2; GTSAM’s default is 1e5). The one-corner-per-tag runs, where equidistant converged, and the section 1 rows were not re-run on f902d39. Nor did we test how much the kb4 fold near the image corners matters downstream. The MVIS authors ran every solve.

Data

TUM-Fisheye: the 15 released MVIS calibrations (two corners per tag, MVIS 197549a; kb4 on BF2M, equidistant on the other lenses) are on the Hub, calibench-vi/datasets → tum-fisheye/results/2026-11-04. The section 2 runs on f902d39 and the one- and all-corner runs behind section 1 are diagnostic runs and are not published. The images are from the Double Sphere paper’s data (Usenko et al., 2018); please cite it.
T265: virig-5imus-4cams, CC BY-NC 4.0, gated; commercial use by request. Please cite the VI-Rig dataset and MVIS.
TUM-VI: TUM-VI.