The effect of antagonistic pleiotropy on the estimation of the average coefficient of dominance of deleterious mutations

We investigate the impact of antagonistic pleiotropy on the most widely used methods of estimation of the average coefficient of dominance of deleterious mutations from segregating populations. A proportion of the deleterious mutations affecting a given studied fitness component are assumed to have an advantageous effect on another one, generating overdominance on global fitness. Using diffusion approximations and transition matrix methods, we obtain the distribution of gene frequencies for nonpleiotropic and pleiotropic mutations in populations at the mutation-selection-drift balance. From these distributions we build homozygous and heterozygous chromosomes and assess the behavior of the estimators of dominance. A very small number of deleterious mutations with antagonistic pleiotropy produces substantial increases on the estimate of the average degree of dominance of mutations affecting the fitness component under study. For example, estimates are increased three- to fivefold when 2% of segregating loci are overdominant for fitness. In contrast, strengthening pleiotropy, where pleiotropic effects are assumed to be also deleterious, has little effect on the estimates of the average degree of dominance, supporting previous results. The antagonistic pleiotropy model considered, applied under mutational parameters described in the literature, produces patterns for the distribution of chromosomal viabilities, levels of genetic variance, and homozygous mutation load generally consistent with those observed empirically for viability in Drosophila melanogaster.