Journal Article

Reliable Detection of Loci Responsible for Local Adaptation: Inference of a Null Model through Trimming the Distribution of FST

Lotterhos, Katie E., Whitlock, Michael C.

2015

Overview

Loci responsible for local adaptation are likely to have more genetic differentiation among populations than neutral loci. However, neutral loci can vary widely in their amount of genetic differentiation, even over the same geographic range. Unfortunately, the distribution of differentiation—as measured by an index such as FST—depends on the details of the demographic history of the populations in question, even without spatially heterogeneous selection. Many methods designed to detect FST outliers assume a specific model of demographic history, which can result in extremely high false positive rates for detecting loci under selection. We develop a new method that infers the distribution of FST for loci unlikely to be strongly affected by spatially diversifying selection, using data on a large set of loci with unknown selective properties. Compared to previous methods, this approach, called OutFLANK, has much lower false positive rates and comparable power, as shown by simulation.

• Publisher: University of Chicago Press