We report the results of infrared imaging photometry of four radio galaxies with redshifts greater than 2. The infrared images show a wide range of morphologies when compared to the optical images. Two objects are quite compact in the IR while one is significantly extended and is closely aligned with its radio source axis and follows the optical morphology. The spectral energy distributions of each galaxy are determined and these too show a wide range of colors, with the more compact objects redder. One object, MRC 0156-252 at z = 2.02, clearly requires a two-component model to reproduce both the rest frame UV/visible morphologies and the spectral energy distribution. The other objects are well fitted two-component models, but single-age models with rapidly decaying star formation provide equally good fits. The implications regarding the redshift of formation and the alignment effect are discussed.