The Patterson-Price-Reich's rule of population structure analysis from genetic marker data

Delineating population structure from the marker genotypes of a sample of individuals is now routinely conducted in the fields of molecular ecology, evolution and conservation biology. Various Bayesian and likelihood methods as well as more general statistical methods (e.g. PCA) have been proposed to detect population structure, to assign sampled individuals to discrete clusters (subpopulations), and to estimate the admixture proportions of each sampled individual. Regardless of the methods, the power of a structure analysis depends on the strength of population structure (measured by FST) relative to the amount of marker information (measured by NL, where N and L are the numbers of sampled individuals and loci respectively). Patterson, Price and Reich (2006) proposed that population structure is unidentifiable when data size D = NL is smaller than 1/FST2 and quickly becomes identifiable easily with an increasing D or FST when D>1/FST2. In this study, I investigated this phase change PPR rule by analysing both simulated genomic data and empirical data by four likelihood admixture analysis methods. The results show that the PPR rule is largely valid, but the accuracy of a structure analysis is also affected by the number of subpopulations K. A more complicated population structure with a larger K requires a larger NLFST2 to resolve accurately. For a given NLFST2 above the PPR threshold value of 1, increasing L and decreasing N is advantageous over increasing N and decreasing L in improving admixture estimation accuracy.