The influence of the epitaxial strain on the structural evolution with temperature of AuNi metastable alloys thin films is investigated. Samples with different initial configurations (codeposited Au1-xNix solid solutions and artificially layered structures) were grown by molecular-beam epitaxy on different (001)-oriented buffer layers (Au, Pt, and Pd). The epitaxial strain was varied by changing (i) the Ni content of the AuNi layer for a given kind of buffer layer, (ii) the nature of the buffer layer for a fixed Ni content, and (iii) the AuNi layer thickness for a fixed Ni content and buffer layer. The structural evolution upon annealing in the 180-300 degrees C temperature range was studied by in situ temperature x-ray diffraction as well as high-resolution electron microscopy. It is shown that a modulated structure develops along the growth direction of the AuNi layer, when the temperature reaches 200-240 degrees C, provided that the residual strain is high enough (>2%). This structure consists of a periodic stacking of 1 Ni-rich plane and 2 or 3 Au-rich planes, depending on the Ni content. The results are explained in terms of a strain-stabilized ordering effect, as supported by energetic calculations based on semiempirical interatomic potentials within the tight-binding scheme.