Color from shape from color: a simple formalism with known light sources

Photometric stereo is a well-known technique for recovering surface normals of a surface but requires three or more images of a surface taken under illumination from different directions. At best, one may dispense with the need for multiple images by using colored lights tuned to camera filters. But a less restrictive paradigm is available that uses the orientation-from-color approach, wherein multiple broadband illuminants impinge on a surface simultaneously. In that method, colors for a Lambertian surface lie on an ellipsoid in color space. The method has been applied mainly to single-color objects, with ellipsoid quadratic-form parameters determined from a large number of pixels. However, recently Petrov and Antonova [Color Res. Appl. 21, 97 (1996)] developed an entirely local approach, useful also for multicolored objects with color uniform in each patch. We investigate to what extent a method such as that of Petrov and Antonova can be applied in the ostensibly simpler situation in which the complex lighting environment is known, i.e., a color photometric stereo situation, with all lights in play at once with only a single image to analyze. We find that, assuming a simple model of color formation, we are able to recover the object colors along with surface normals, using only a single image. Because we immerse the object in a known lighting environment, we show that only half of the equations utilized by Petrov and Antonova are actually needed, making the method more stable. Nevertheless, solutions do not exist at every pixel; instead we may determine a best estimate of patch color, using a robust estimator, and then apply that estimate throughout a patch. Results are shown to be quite good compared with ground truth. The simple color model can often be made to hold more exactly by transforming the color space into one corresponding to spectrally sharpened sensors, which are a matrix transform away from the actual camera sensors. In our study the reliability and accuracy of the normal vector and of the surface color recovery algorithm are improved by this straightforward transformation. (C) 2000 Optical Society of America [S0740-3232(00)01008-5].