An extension of Schwarzschild's galaxy-building technique is presented that enables one to build Schwarzschild models with known distribution functions (DFs). The new extension makes it possible to combine a DF that depends only on classical integrals with orbits that respect non-classical integrals. With such a combination, Schwarzschild's orbits are used only to represent the difference between the true galaxy DF and an approximating classical DF. The new method is used to construct a dynamical model of the inner Galaxy. The model is based on an orbit library that contains 22 168 regular orbits. The model aims to reproduce the three-dimensional mass density of Binney, Gerhard & Spergel, which was obtained through deprojection of the COBE surface photometry, and to reproduce the observed kinematics in three windows - namely Baade's Window with (l,b)=(1 degrees,-4 degrees) and two off-axis fields at (8 degrees, 7 degrees) and (12 degrees, 3 degrees). The viewing angle is assumed to be 20 degrees to the long axis of the bar and the pattern speed is taken to be 60 km s(-1) kpc(-1). The model fits essentially all the available data within the innermost 3 kpc. The axial ratio and the morphology of the projected density contours of the COBE bar are recovered to excellent accuracy within corotation. The kinematic quantities - the line-of-sight streaming velocity and velocity dispersion, as well as the proper motions when available - are recovered, not merely for the fitted fields at (1 degrees, -4 degrees) and (8 degrees, 7 degrees), but also for three new fields at (84, -6 degrees), (121, -167) and (-114, 181). The dynamical model deviates most from the input density close to the Galactic plane just outside corotation, where the deprojection of the surface photometry is suspect. The dynamical model does not reproduce the kinematics at the most distant window, (12 degrees, 3 degrees), where disc contamination of the data may be severe. Maps of microlensing optical depth are presented both for randomly chosen stars and for stars that belong to individual components within the model. While the optical depth to a randomly chosen star in Baade's Window is half what measurements imply, the optical depth to stars in a particular component can be as high as the measured values. The contributions to the optical depth towards randomly chosen stars from lenses in different components are also given.