Submit results
Use this page if you want results from your calibration tool next to MVIS, Kalibr and Basalt on the leaderboard.
How the leaderboard is built today
Section titled “How the leaderboard is built today”- Who runs it: we run MVIS, Kalibr and Basalt ourselves on the same
recordings. The current numbers are from the batch of 2026-11-10 (batch
2026-11-10a). - What it shows: each calibration parameter per tool, as the median and spread over the recordings of one test family (a group of recordings of one rig, solved with one setup). Real recordings have no ground truth, so a gap between tools shows where they disagree, not which one is right. See Methodology.
- Where the data lives: every record is public in the Hugging Face dataset
calibench-vi/resultsunder the CC BY 4.0 licence.
The results repository has this layout:
| Path | Contents |
|---|---|
<Tool>/<batch>/<family>.json | one record per tool and test family: per method, the number of recordings, the median reprojection error, and each parameter’s median, spread, count and status |
configs/<family>/<Tool>/ | the configuration files behind each record |
schemas/ | the JSON schema of a record |
datasets.json, sensors.json | the dataset and sensor catalogues |
families.json | datasets to families, with each family’s recordings, sensors and caveats |
Compare your tool now
Section titled “Compare your tool now”You do not need us to compare your tool against the leaderboard.
- Get the data. Download the recordings from
calibench-vi/datasets(see Datasets). The VI-Rig sets are gated: accept the terms on the Hugging Face page first. - Use the same recordings. Take the recording list of a family from
families.jsonincalibench-vi/results. - Run your tool on those recordings.
- Compare your values with the MVIS, Kalibr and Basalt records of that family and batch.
Send us your results
Section titled “Send us your results”Open an issue in
openmvis/calibench-submissions.
Include:
- Tool: name, version and commit hash, and a link to the code or paper.
- Configs: the exact configuration files and command-line flags, per dataset. Attach the files; do not describe them in prose.
- Recordings: the dataset and recordings you ran, and which ones you used for calibration.
- Outputs: your calibration outputs, not scores: intrinsics, distortion, extrinsics and time offsets per run. Add each parameter’s standard deviation (σ) if your tool reports it.
- Statistic: for any metric you report yourself, its exact definition, for example “median reprojection error over all corners of all frames”.
- Runtime and hardware: wall-clock time per dataset, CPU, RAM, GPU if used, and operating system.
- Licence: a statement that your outputs may be published under CC BY 4.0.
Rotations: give them as the log(R) rotation vector in radians, or as a rotation matrix or quaternion. Do not give roll, pitch and yaw angles.
We reply on the issue. When the new submission process opens, this page and the
README of openmvis/calibench-submissions describe it.