Evaluation of Interest Point Detectors
CORDELIA SCHMIDROGER MOHRCHRISTIAN BAUCKHAGE
Establishes a rigorous benchmarking framework using repeatability rate and information content to quantitatively determine which interest point detectors provide the highest geometric stability and feature distinctiveness across varying imaging conditions.
Many computer vision applications—such as image matching, 3D scene reconstruction, and automated object recognition—depend heavily on the reliable extraction of low-level visual features known as interest points. Historically, evaluating these detectors relied on subjective human visual inspections, narrow feature models, or application-specific pipelines that were difficult to generalize. Because the choice of detector dramatically alters system performance, developers need objective, generalizable benchmarks to choose the best algorithms for multi-image processing tasks.
The article establishes two rigorous, automated evaluation criteria—repeatability rate and information content—to quantitatively assess and rank leading interest point detectors across realistic operating conditions.
The evaluation framework tests detectors across two fundamental dimensions. First, repeatability measures the geometric stability of detected points across changing physical imaging conditions, including image rotations, scale changes, lighting variations, viewpoint shifts, and camera sensor noise. Geometric ground truth is established on static scenes using projective transformations calculated via independently projected calibration patterns. Second, information content evaluates the distinctiveness of local image patterns around detected points using entropy measurements across a dataset of 1,000 diverse images (aerial scenes, paintings, and objects) to quantify how well these features prevent ambiguity during matching.
The experimental results identify an improved version of the Harris detector—implemented with stable Gaussian derivative filters—as the top-performing method overall. Under image rotations, this improved detector maintains nearly a 100% repeatability rate within a 1.5-pixel error margin, completely eliminating the 45-degree performance dips seen in standard discrete implementations. Across viewpoint changes, it consistently maintains repeatability above 60%, significantly outperforming contour-based and model-based alternatives. In distinctiveness evaluations, the improved Harris detector achieved the highest entropy score (6.05), followed closely by the biologically inspired Heitger detector (5.94), whereas contour-only detectors scored noticeably lower and random points produced an entropy of only 3.30 (making false feature collisions roughly 15.6 times more likely). Across all detectors, scale changes proved to be the most severe vulnerability, with repeatability collapsing rapidly when image scale varied by a factor of 1.5 to 2.0 or greater.
These findings provide clear practical guidelines for system designers. Selecting an interest point detector that maximizes both repeatability and information content substantially lowers the risk of feature mismatching, reduces downstream computational overhead, and improves tracking stability. The results also show that relying strictly on image contours degrades overall feature distinctiveness, and using unaligned directional derivative filters introduces hidden geometric vulnerabilities when images rotate.
For immediate deployment in matching and tracking pipelines, the article recommends using the improved Gaussian-based Harris detector. To overcome the universal limitation of scale sensitivity, engineering teams should incorporate these interest point operators into multi-scale frameworks or implement explicit scale-selection mechanisms before applying them to scenes with large zoom or distance changes.
Confidence in these findings is high for planar and general surface matching under rotation, lighting changes, and moderate viewpoint variations. However, decision-makers should note that the geometric ground-truth validation is mathematically constrained to planar scenes to avoid confounding detector performance with physical occlusions and shadow boundaries that naturally occur in complex 3D environments.
- Paper: Local Grayvalue Invariants for Image Retrieval, Cordelia Schmid et al. (1997). Introduces local grayvalue invariants at interest points for image matching and retrieval, establishing the foundational local-feature paradigm evaluated and benchmarked in the source paper.
- Paper: Feature Detection with Automatic Scale Selection, Tony Lindeberg (1998). Develops the principles of automatic scale selection and differential scale-space feature detection that underpin multi-scale interest point formulation and evaluation.
- Paper: Scale & Affine Invariant Interest Point Detectors, K. Mikolajczyk et al. (2004). Builds directly on the repeatability evaluation methodology to develop and validate scale- and affine-invariant interest point detectors such as Harris-Laplace.
- Paper: A Comparison of Affine Region Detectors, K. Mikolajczyk et al. (2005). Extends the source's evaluation framework to affine-covariant region detectors using homography-based repeatability and descriptor matching benchmarks.
- Paper: A performance evaluation of local descriptors, Krystian Mikolajczyk et al. (2005). Complements interest point detector evaluation by establishing a systematic benchmark for the local descriptors extracted around those detected points.
- Paper: Distinctive Image Features from Scale-Invariant Keypoints, David G. Lowe (2004). Presents scale-invariant feature transform (SIFT) keypoint detection and description, heavily relying on repeatability and distinctiveness criteria established in earlier feature evaluation work.
- Paper: Machine Learning for High-Speed Corner Detection, Edward Rosten et al. (2006). Uses the repeatability metric formalized in interest point evaluation benchmarks to design and evaluate the high-speed FAST corner detector.
- Paper: SURF: Speeded Up Robust Features, Herbert Bay et al. (2006). Introduces SURF as an accelerated scale- and rotation-invariant detector-descriptor pair evaluated against standard repeatability and distinctiveness protocols.
- Paper: On Space-Time Interest Points, I. Laptev (2005). Generalizes the spatial interest point concept and scale selection principles into the spatio-temporal domain for video event representation.
- Paper: SuperPoint: Self-Supervised Interest Point Detection and Description, Daniel DeTone et al. (2017). Revisits repeatable interest point detection in a modern deep learning framework using self-supervised homographic adaptation.
