The maintenance of phenotypic and genetic variation in threshold traits by frequency-dependent selection

Many traits are phenotypically dimorphic but determined by the action of many loci, the phenotype being a result of a threshold of sensitivity. Quantitative genetic analysis has shown that generally there is considerable additive genetic variation for the trait, the average heritability being 0.52. In numerous cases threshold traits have been shown, or are assumed, to be under frequency-dependent selection; examples include satellite-territorial behaviour, sex-determination, wing dimorphism and trophic dimorphism. In this paper I investigate the potential for frequency-dependent selection to maintain both phenotypic and additive genetic variation in threshold traits. The qualitative results are robust to the particular form of the frequency-dependent selection function. The equilibrium proportion is more or less independent of population size but the heritability increases with population size, typically approaching its maximal value at a population size of 5000, when the mutation rate is 10(-4). A tenfold decrease in the mutation rate requires an approximate doubling of the population size before an asymptotic value is approached. Thus frequency-dependent selection can account for both the existence of two morphs in a population and the observed levels of heritability. It is also shown, both via simulation and theory, that the quantitative genetic model and a simple phenotypic analysis predict the same equilibrium morph proportion.