A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
S. SeitzB. CurlessJ. DiebelD. ScharsteinR. Szeliski
Establishes a standardized benchmark and evaluation methodology for multi-view stereo reconstruction by providing calibrated image datasets with high-precision laser-scanned ground truth alongside a comprehensive algorithmic taxonomy.
This paper addresses the absence of standardized, calibrated multi-view image datasets with accurate ground-truth 3D models, which had blocked direct quantitative comparisons among reconstruction algorithms and slowed progress in the field. Without such benchmarks, researchers could not reliably identify strengths, weaknesses, or priority areas for improvement, unlike the situation in binocular stereo where shared test data had already accelerated gains.
The work set out to create the first public collection of high-quality multi-view datasets registered to laser-scanned ground truth, together with a taxonomy of algorithm properties and an evaluation framework that measures both geometric accuracy and surface completeness.
The authors first surveyed existing techniques and organized them by scene representation, photo-consistency measure, visibility handling, shape priors, reconstruction strategy, and initialization needs. They then captured roughly 300–360 calibrated images per object for two subjects using a precision robotic gantry, produced reference meshes via dense laser scanning, and refined the alignment of each mesh to the images by minimizing photo-consistency error. Six leading algorithms were run on full-hemisphere, ring, and sparse-ring subsets of the data; accuracy was reported as the distance within which 90 percent of reconstructed points lie from the ground truth, and completeness as the fraction of ground-truth points recovered within a 1.25 mm tolerance.
The evaluation shows that several methods reach sub-millimeter accuracy from standard video-resolution images, with the best result placing 90 percent of points within 0.36 mm on the full temple set. Accuracy remains high across view counts for textured objects but varies more for low-texture surfaces, where regularization influences outcomes. Completeness is generally strong when silhouettes are available, yet drops for methods that leave holes in uncertain regions. Offsets of several tenths of a millimeter between independently produced models required explicit alignment before comparison.
These results demonstrate that current multi-view stereo techniques can already deliver models accurate enough for many practical uses, while also revealing that performance still depends on scene texture, view density, and the use of silhouette constraints. The availability of common test data and repeatable metrics should therefore focus future effort on the remaining gaps, such as handling specular surfaces and operating without silhouettes.
The authors plan to release additional datasets with specularities and no silhouettes, to acquire higher-resolution imagery paired with industrial-grade ground truth, and to keep the evaluation open for new submissions. Researchers should treat the current numbers as a baseline rather than a final ranking, because the study covers only algorithms that supplied results by the original deadline and assumes largely Lambertian reflectance.
- Paper: A Flexible New Technique for Camera Calibration, Zhengyou Zhang (2000). Zhang's camera calibration technique provides the essential geometric foundations and planar grid methods required to acquire calibrated multi-view datasets.
- Paper: Light field rendering, Marc Levoy et al. (1996). Light field rendering establishes the foundational image-based representations and multi-view sampling principles that pre-date modern multi-view stereo evaluation.
- Paper: A volumetric method for building complex models from range images, B. Curless et al. (1996). Curless and Levoy's volumetric range-merging algorithm supplies the crucial surface reconstruction and distance integration techniques that multi-view stereo algorithms evaluate.
- Paper: An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision, Yuri Boykov et al. (2001). Min-cut/max-flow energy minimization algorithms provide the core optimization machinery utilized by many of the stereo reconstruction pipelines evaluated in the source.
- Paper: Structure-from-Motion Revisited, Johannes L. Schönberger et al. (2016). COLMAP builds upon the foundational multi-view stereo principles and benchmark datasets established in the source to deliver a robust, end-to-end structure-from-motion pipeline.
- Paper: NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction, Peng Wang et al. (2021). NeuS extends traditional multi-view reconstruction into the neural implicit domain, utilizing datasets like Middlebury and DTU to evaluate high-fidelity surface estimation.
- Paper: 3D Gaussian Splatting for Real-Time Radiance Field Rendering, Bernhard Kerbl et al. (2023). 3D Gaussian Splatting advances 3D scene reconstruction from multi-view imagery, shifting from traditional geometric stereo and voxel methods to real-time radiance field rendering.
