Acquiring linear subspaces for face recognition under variable lighting
Kuang-chih LeeJ. HoDavid Kriegman
Shows that low-dimensional linear subspaces for face recognition under variable lighting can be constructed directly from five to nine real images taken under specific point-source directions, eliminating the need for 3D reconstruction or large training datasets.
Lighting variation poses a central challenge for robust face recognition systems, as images of the same face can differ dramatically depending on the number, direction, and intensity of light sources. Prior theoretical work established that these variations can be modeled by low-dimensional linear subspaces, including the illumination cone and its nine-dimensional harmonic approximation, yet computing or acquiring the required basis images has typically demanded either dense image sets, 3D surface reconstruction, or physically unrealizable lighting.
The article set out to determine whether small numbers of physically realizable single distant light sources could be chosen so that the images they produce span a subspace that closely approximates the illumination cone and supports accurate recognition.
The authors developed two greedy selection algorithms that operate on a large set of candidate directions sampled uniformly over the sphere or hemisphere. One algorithm maximizes a similarity measure based on principal angles between the candidate subspace and the harmonic subspace; the second additionally maximizes the solid angle of the intersection between the candidate subspace and the illumination cone. Both algorithms were run on 3D face models from the Yale Face Database B to produce nested sequences of subspaces from one to nine dimensions. The resulting per-person configurations proved qualitatively similar, allowing a single “universal” configuration of nine directions to be computed by averaging the objective function across individuals.
When subspaces spanned by images taken under the universal configuration were used for recognition on the Yale Face Database B, error rates fell to zero with nine images and remained below 1 percent with as few as five images. On the larger extended Yale set the five-dimensional universal subspace produced a 0.2 percent error rate, an order of magnitude better than the median performance of 16,000 randomly chosen five-light configurations. Comparable results held on the CMU PIE database. Subspaces of dimension five and higher performed well even when test images contained strong ambient illumination, and performance improved further as additional ambient sources were introduced.
These findings indicate that carefully chosen sets of five to nine real images can replace both dense training collections and intermediate 3D reconstruction steps, thereby lowering the cost and complexity of enrolling individuals in controlled environments such as security checkpoints or licensing offices. The approach also suggests that, when lighting statistics are known in advance, even fewer training images may suffice.
The principal limitations are the assumption of approximately Lambertian reflectance, the use of convex-face models that omit some cast-shadow effects, and reliance on greedy rather than exhaustive search for the optimal directions. Nevertheless, the consistency of the discovered configurations across subjects and databases, together with the large performance gap relative to random selections, supports high confidence that the reported universal configurations are effective and practical for real-world face recognition under variable lighting.
- Paper: Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, Peter N. Belhumeur et al. (1996). This paper establishes the foundational linear subspace framework for Lambertian surfaces under varying illumination that the source directly seeks to acquire from sparse physical lighting.
- Paper: A Morphable Model For The Synthesis Of 3D Faces, Volker Blanz et al. (1999). This work introduces the 3D morphable model methodology used to reconstruct the 3D surface geometry necessary for analyzing face illumination variations.
- Paper: Face Recognition Based on Fitting a 3D Morphable Model, Volker Blanz et al. (2003). It provides the 3D model-fitting paradigm across pose and illumination that the source replaces with a sparse physical image acquisition strategy.
- Paper: Two-dimensional PCA: a new approach to appearance-based face representation and recognition, Jian Yang et al. (2004). It provides foundational linear projection techniques for appearance-based face representation under illumination and expression shifts.
- Paper: Face Recognition: Features Versus Templates, R. Brunelli et al. (1993). It establishes classic baseline dimensionality limits and representation trade-offs in linear face subspaces.
- Paper: Robust Face Recognition via Sparse Representation, John Wright et al. (2009). This work extends linear subspace modeling for face recognition by formulating sparse representation over illumination dictionaries to achieve robustness against extreme lighting and occlusion.
- Paper: Face recognition using Laplacianfaces, Xiaofei He et al. (2005). This paper advances linear subspace face recognition under lighting variations by preserving local manifold geometry through locality preserving projections.
- Paper: Incremental Learning for Robust Visual Tracking, David A. Ross et al. (2008). It applies the concept of low-dimensional appearance subspaces dynamically by incrementally updating the linear basis to track objects under continuous lighting changes.
- Paper: Deep Retinex Decomposition for Low-Light Enhancement, Chen Wei et al. (2018). This work generalizes physical lighting decomposition by using deep Retinex networks to separate illumination and reflectance components under severe low-light conditions.
