The polarization of light scattered out of the plane of incidence was measured from rough and microrough silicon, polished fused silica and glass ceramic, and ground and polished black glass. The measurement results are in excellent agreement with models for scattering from microroughness or subsurface defects, demonstrating that the polarization of scattered light can be used to distinguish between microroughness and subsurface defects.