Matching with PROSAC - progressive sample consensus
Ondřej ChumJiri Matas
Proposes a sample consensus algorithm that accelerates correspondence matching by orders of magnitude over RANSAC through progressive sampling of tentative matches ordered by similarity, while retaining identical worst-case convergence guarantees.
Reliable feature matching between multiple images is a foundational requirement in computer vision tasks such as 3D reconstruction, stereo matching, and motion tracking. Traditional pipelines use standard random sample consensus (RANSAC) to filter out incorrect matches (outliers) from correct geometric alignments (inliers). However, standard RANSAC treats all candidate correspondences as equally probable, drawing samples uniformly across the entire dataset. When images contain large fractions of outliers caused by occlusions, texture repetition, or significant viewpoint changes, standard random sampling becomes prohibitively slow and computationally expensive.
The article evaluates Progressive Sample Consensus (PROSAC), a robust estimation method designed to dramatically accelerate geometric model fitting. The main objective is to demonstrate that sampling candidate matches in order of their initial descriptor similarity yields massive speedups while maintaining the same statistical reliability guarantees as standard RANSAC.
To establish its findings, the article evaluates PROSAC across challenging wide-baseline image pairs featuring repetitive patterns, depth discontinuities, occlusions, and independent foreground motions. The approach uses existing local similarity measures (such as SIFT descriptor ratios and transform coefficient distances) to order correspondences from highest to lowest quality. Instead of sampling uniformly, PROSAC samples from progressively larger subsets of the top-ranked candidates, dynamically balancing early deterministic testing of high-quality matches with gradual convergence toward standard uniform sampling. The experiments compare sample counts and wall-clock execution times across benchmark scenes against traditional RANSAC.
The analysis reveals three key findings. First, the foundational assumption that match similarity predicts true correctness better than random guessing held across all tested datasets, with inlier concentration declining steadily toward lower-ranked matches. Second, PROSAC delivered substantial speed improvements—often reducing computation time by a factor of 100 or more. In one benchmark scene, PROSAC required an average of 9 samples (0.06 seconds) to resolve the geometric model compared to 106,534 samples (10.76 seconds) for RANSAC. Third, PROSAC successfully recovered geometric alignments in extreme noise environments where RANSAC failed completely, such as a scene requiring an estimated 84 million RANSAC trials that PROSAC resolved in 3,576 samples (0.76 seconds).
These findings have significant practical implications for real-time computer vision systems, motion segmentation, and autonomous tracking pipelines. By finding valid models early within high-confidence subsets, PROSAC substantially reduces latency, computational cost, and hardware resource demands. Furthermore, it eliminates the operational burden of manually fine-tuning conservative similarity thresholds, making computer vision pipelines significantly more robust against noisy or sparse candidate pools.
Organizations developing feature matching and multi-view vision systems should adopt PROSAC in place of standard RANSAC for geometric model estimation. When integrating the method, developers can safely relax initial feature-filtering thresholds to capture more true matches without risking catastrophic performance degradation from added outliers.
Confidence in these results is high, as PROSAC mathematically converges to standard RANSAC in worst-case scenarios where similarity ordering is purely random. Tests using randomly permuted data confirmed that PROSAC maintains performance at or slightly above RANSAC even when quality rankings provide zero predictive value. However, users should note that the speed advantage diminishes if local similarity metrics fail to correlate with correct matches, such as in scenes dominated by extreme repetitive textures or independent foreground motions that share high visual similarity.
- Paper: Distinctive Image Features from Scale-Invariant Keypoints, David G. Lowe (2004). It introduces the SIFT feature detector and descriptor, whose similarity metric and nearest-neighbor distance ratios provide the foundational sorting order that PROSAC exploits for prioritized sampling.
- Paper: An efficient solution to the five-point relative pose problem, David Nister (2004). It establishes the efficient minimal 5-point relative pose solver used within robust hypothesis generation pipelines that PROSAC accelerates.
- Paper: In Defense of the Eight-Point Algorithm, Richard I. Hartley (1997). It provides the normalized eight-point algorithm for estimating epipolar geometry and fundamental matrices, serving as a classical geometric model fitted during robust sample consensus.
- Paper: Scale & Affine Invariant Interest Point Detectors, Krystian Mikolajczyk et al. (2004). It details scale- and affine-invariant interest point detectors that provide the candidate visual correspondences necessary for wide-baseline geometric model estimation.
- Paper: Evaluation of Interest Point Detectors, CORDELIA SCHMID et al. (2000). It establishes foundational criteria for measuring feature detector repeatability and distinctiveness across varying viewpoints prior to geometric correspondence matching.
- Paper: Towards Linear-Time Incremental Structure from Motion, Changchang Wu (2013). It incorporates scale-prioritized, preemptive feature matching within large-scale structure-from-motion pipelines to achieve linear-time incremental 3D reconstruction.
- Paper: Structure-from-Motion Revisited, Johannes L. Schönberger et al. (2016). It incorporates robust geometric verification and recursive sample consensus into a comprehensive, modern structure-from-motion reconstruction system (COLMAP).
- Paper: Accurate, Dense, and Robust Multiview Stereopsis, Yasutaka Furukawa et al. (2010). It builds upon robustly estimated geometric camera alignments to perform dense, patch-based multi-view stereo reconstruction.
- Paper: SuperPoint: Self-Supervised Interest Point Detection and Description, Daniel DeTone et al. (2017). It develops a learned interest point detector and descriptor whose high-quality match scores and candidate correspondences can be directly paired with progressive consensus estimators.
- Paper: LIFT: Learned Invariant Feature Transform, Kwang Moo Yi et al. (2016). It replaces hand-crafted features with end-to-end learned invariant representations to produce superior initial correspondence rankings for geometric robust estimation.
- Paper: Learning to compare image patches via convolutional neural networks, Sergey Zagoruyko et al. (2015). It trains deep convolutional networks to accurately score patch similarity, providing the predictive match rankings that prioritized sampling consensus methods depend on.
- Paper: DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo, Engin Tola et al. (2010). It designs an efficient dense local descriptor optimized for wide-baseline matching where geometric verification is required across severe perspective changes.
