This paper presents a method, which integrates image knowledge and Light Detection And Ranging (LiDAR) point cloud data for urban digital terrain model (DTM) and digital building model (DBM) generation. The DBM is an Object-Oriented data structure, in which each building is considered as a building object, i.e., an entity of the building class. The attributes of each building include roof types, polygons of the roof surfaces, height, parameters describing the roof surfaces, and the LiDAR point array within the roof surfaces. Each polygon represents a roof surface of building. This type of data structure is flexible for adding other building attributes in future, such as texture information and wall information. Using image knowledge extracted, we developed a new method of interpolating LiDAR raw data into grid digital surface model (DSM) with considering the steep discontinuities of buildings. In this interpolation method, the LiDAR data points, which are located in the polygon of roof surfaces, first are determined, and then interpolation via planar equation is employed for grid DSM generation. The basic steps of our research are: (1) edge detection by digital image processing algorithms; (2) complete extraction of the building roof edges by digital image processing and human-computer interactive operation; (3) establishment of DBM; (4) generation of DTM by removing surface objects. Finally, we implement the above functions by MS VC++ programming. The outcome of urban 3D DSM, DTM and DBM is exported into urban database for urban 3D GIs, (C) 2004 Elsevier Ltd. All rights reserved.