Skip to content

Submit results

Use this page if you want results from your calibration tool next to MVIS, Kalibr and Basalt on the leaderboard.

  • Who runs it: we run MVIS, Kalibr and Basalt ourselves on the same recordings. The current numbers are from the batch of 2026-11-10 (batch 2026-11-10a).
  • What it shows: each calibration parameter per tool, as the median and spread over the recordings of one test family (a group of recordings of one rig, solved with one setup). Real recordings have no ground truth, so a gap between tools shows where they disagree, not which one is right. See Methodology.
  • Where the data lives: every record is public in the Hugging Face dataset calibench-vi/results under the CC BY 4.0 licence.

The results repository has this layout:

PathContents
<Tool>/<batch>/<family>.jsonone record per tool and test family: per method, the number of recordings, the median reprojection error, and each parameter’s median, spread, count and status
configs/<family>/<Tool>/the configuration files behind each record
schemas/the JSON schema of a record
datasets.json, sensors.jsonthe dataset and sensor catalogues
families.jsondatasets to families, with each family’s recordings, sensors and caveats

You do not need us to compare your tool against the leaderboard.

  1. Get the data. Download the recordings from calibench-vi/datasets (see Datasets). The VI-Rig sets are gated: accept the terms on the Hugging Face page first.
  2. Use the same recordings. Take the recording list of a family from families.json in calibench-vi/results.
  3. Run your tool on those recordings.
  4. Compare your values with the MVIS, Kalibr and Basalt records of that family and batch.

Open an issue in openmvis/calibench-submissions. Include:

  • Tool: name, version and commit hash, and a link to the code or paper.
  • Configs: the exact configuration files and command-line flags, per dataset. Attach the files; do not describe them in prose.
  • Recordings: the dataset and recordings you ran, and which ones you used for calibration.
  • Outputs: your calibration outputs, not scores: intrinsics, distortion, extrinsics and time offsets per run. Add each parameter’s standard deviation (σ) if your tool reports it.
  • Statistic: for any metric you report yourself, its exact definition, for example “median reprojection error over all corners of all frames”.
  • Runtime and hardware: wall-clock time per dataset, CPU, RAM, GPU if used, and operating system.
  • Licence: a statement that your outputs may be published under CC BY 4.0.

Rotations: give them as the log(R) rotation vector in radians, or as a rotation matrix or quaternion. Do not give roll, pitch and yaw angles.

We reply on the issue. When the new submission process opens, this page and the README of openmvis/calibench-submissions describe it.