Hard coatings deposited by plasma-assisted vapor deposition methods based on the transition metal nitrides are nowadays widely applied to reduce wear and corrosion of tools. In the last two decades, tremendous advances have been achieved in the development of deposition and application technology of these coatings. The methods to be applied to optimize coating properties are known from materials science, however, only in the last few years significant attempts have been made to use them for designing coatings showing the properties required. In particular, plasma-assisted vapor deposition processes are ideally suited to control the microstructure and composition of hard coatings. Coatings deposited by these processes are often in a non-equilibrium state, e.g., characterized by high compressive stresses, small grain sizes in the nanometer range, or metastable phases, which determine their unique properties like superhardness, toughness, or low friction. The aim of this paper is to present and discuss several mechanisms of hardness enhancement in single-phase and multi-phase coatings based on transition metal nitrides, carbides and borides. These hardening effects include the role of strain hardening by high defect densities obtained by high-energy ion irradiation during deposition, grain size refinement to the nano-scale, solid-solution hardening by alloying elements, and age-hardening by coherent domains. Special emphasis will be laid on the magnitude of these effects in determining coating hardness and on their thermal stability.