Extreme selection strategies in gene mapping studies of oligogenic quantitative traits do not always increase power

It is well known that obtaining adequate statistical power to detect linkage to or association with genes for complex quantitative traits can be very difficult, In response, investigators have developed a number of power-enhancing strategies that consider restraints such as genotyping (and/or phenotyping) costs. In the context of both association and sib pair linkage studies of quantitative traits, one of the most widely discussed techniques is the selective sampling of phenotypically extreme individuals. Several papers have demonstrated that such extreme sampling can markedly increase power (under certain circumstances). However, the parenthetical phrase in the previous sentence has generally not been made explicit and it appears to be implied that the more phenotypically extreme the individuals, the more power one has. In this paper, we show by simulation that this is not true under all circumstances. In particular, we show that under oligogenic models, where some biallelic quantitative trait loci (QTLs) have markedly asymmetric allele frequencies and large mean displacement among genotypes, and others have less asymmetric allele frequencies and smaller mean displacement among genotypes, power to detect linkage to or association with the latter QTL can actually decrease by sampling more extreme sib pairs. This suggests that more extreme sampling is not always better. The 'optimal' sampling scheme may depend on both what one suspects the underlying genetic architecture to be and which of the oligogenic QTL one has greatest interest in detecting.