Let sigma be an automorphism of a finite group G and suppose that sigma fixes every element of G that has prime order or order 4. The main result of this paper shows that the structure of the subgroup H = [G, sigma] is severely limited in terms of the order n of sigma. In particular, H has exponent dividing n and it is nilpotent of class bounded in terms of n.