Thin films of the M(n+1)AX(n) (MAX) phase (M: early transition metal; A: A- group element; X: C and/ or N; n= 1 - 3) Ti2AlN were epitaxially grown onto single-crystal MgO (111) and MgO (100) substrates by dc reactive magnetron cosputtering from Ti and Al targets in an Ar/N-2 gas mixture at a temperature of 690 degrees C. To promote the nucleation of the MAX phase, a fcc (Ti0.63Al0.37) N seed layer was deposited before changing to Ti2AlN growth parameters. The nucleation processes have been studied by real-time in situ specular x-ray reflectivity. Independent of substrate orientation, the seed layer shows no roughening until its final thickness of approximately 100 angstrom, indicating pseudomorphic layer-by-layer growth. The MAX phase shows heteroepitaxial layer-by-layer growth on MgO (111), with increased surface roughening up to approximately 200 angstrom whereas on MgO (100) the growth mode changes to Volmer-Weber-type already after three monolayers. X-ray scattering in Bragg-Brentano geometry of the final, approximately 1000 angstrom thick, Ti2AlN film reveals lattice parameters of c = 13.463 angstrom and a = 2.976 angstrom on the MgO (111) substrate and c = 13.740 angstrom and a = 2.224 angstrom on the MgO (100) substrate. From pole figure measurements the orientational relationship between film and substrate lattice was determined to be MgO {111}< 110 >//Ti2AlN{1012}< 1210 >, regardless of the substrate orientation. This tilted, nonbasal-plane growth leads to a threefold grain orientation of Ti2AlN along the MgO < 110 > directions and a polycrystalline morphology confirmed by cross-sectional transmission electron microscopy. The growth can be assumed to take place in a lateral step-flow mode, i.e., emerging low surface free-energy (0001) planes, on which arriving atoms can diffuse until finding a step where they are bound to A facets. This growth process is irrespective of orientational relationship between substrate and film. However, in the present low-temperature case the partitioning of arriving Al and Ti atoms during nucleation is suppressed, which as a result of interfacial adaptation between substrate and film induces standing a-type planes during growth. (c) 2006 American Institute of Physics.