Eigenfaces work as "primary faces." Just as any color can be created by mixing primary colors, any facial image can be built by adding together, with different intensities of light, a relatively small number of eigenfaces (about 100 is enough to identify virtually any person).

With its set of 100 eigenfaces, the computer can now easily analyze all the faces in its facebase, each of which can be expressed as a combination of the eigenfaces—more of some, less of others. The eigenfaces are like filters that allow the computer to see just one aspect of a face at a time. They are also a sort of shorthand for describing just how each face differs from the average.

The system functions by projecting face irnagcs onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes. ears, and noses.