Data processing algorithms for generating textured 3D building facade meshes from laser scans and camera images

In this paper, we develop a set of data processing algorithms for generating textured facade meshes of cities from a series of vertical 2D surface scans, obtained by a laser scanner while driving on public roads under normal traffic conditions. These processing steps are needed to cope with imperfections and non-idealities inherent in laser scanning systems such as occlusions and reflections from glass surfaces. The driven path is cut into easy-to-handle quasi-linear segments with approximately straight driving direction. The vertical scans in each segment are recorded in a sequential topological order, and are transformed into a depth image; if available, additional 3D data from other modalities such as stereo vision is also taken into consideration. Dominant building structures are detected in the depth images, and points are classified into foreground and background layers. Large holes in the background layer, caused by occlusion from foreground layer objects, are filled in by planar or horizontal interpolation. The depth image is further processed by removing isolated points and filling remaining small holes, to obtain a textured surface mesh. We apply the above steps to a large set of data with several million 3D points, and show photorealistic texture mapped 3D models.