The optical-IR images of the northern and southern Hubble Deep Fields are used to measure the spectral and density evolution of early-type galaxies. The mean spectral energy distribution is found to evolve passively toward a mid-F star-dominated spectrum by z similar to 2, becoming more sharply peaked around the 4000 Angstrom break. We demonstrate with realistic simulations that hotter elliptical galaxies would be readily visible if evolution progressed blueward and brightward at z > 2, following a standard initial mass function (IMF). The color distributions are best fitted by a "red" IMF deficient above similar to 2 M-circle dot and with a spread of formation in the range 1.5 < z(f) < 2.5. Traditional age dating is spurious in this context; a distant elliptical can be young but appear red, with an apparent age greater than 3 Gyr independent of its formation redshift. Regarding density evolution, we demonstrate that the sharp decline in numbers claimed at z > 1 results from a selection bias against distant red galaxies in the optical, where the flux is too weak for morphological classification, but is remedied with relatively modest IR exposures that reveal a roughly constant space density to z similar to 2, with 32 and 16 elliptical galaxies detected above and below z = 1, respectively. We point out that the lack of high-mass star formation inferred here and the requirement of metals implicates cooling flows of preenriched gas in the creation of the stellar content of spheroidal galaxies. Deep-field X-ray images will be very helpful in examining this possibility.