<p>I try to understand how to chose the optimized pi_hat parameter for a dataset. In many articles, they chose 0.2 as pi_hat, and everything above that is considered to be cryptic relatedness or duplicates.</p>

<p>I've tested IBD on HapMap, the files I use can be found here: <a href='ftp://ftp.ncbi.nlm.nih.gov/hapmap/genotypes/2009-01_phaseIII/plink_format/'>ftp://ftp.ncbi.nlm.nih.gov/hapmap/genotypes/2009-01_phaseIII/plink_format/</a>. I first remove all annotated offspring from HapMap. Then I peform IBD to see if it still finds samples with cryptic relatedness to each other. The steps I peform are the following (in PLINK):</p>

<pre><code>1) LD-prune:

plink --file hapmap --indep-pairwise 50 5 0.2
plink --file hapmap --extract plink.prune.in --recode --out hapmap_pruned

(2) IBD:

plink --file hapmap_pruned --genome --min 0.2
</code></pre>

<p>The results shows that many cryptic related samples can be found with a pi_hat of 0.2 as threshold, even if all offspring were initially removed. My question is, is this a normal behavior? Or should one increase the pi_hat? How to find out a "good" pi_hat for a custom dataset?</p>

<p>This is normal behavior for the HapMap data set. See <a href='http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0049575'>Stevens et al.</a> and a few of the earlier papers he cites that try to identify un-annotated relationship in HapMap. The Stevens paper is using Cotterman coefficients (K1,K2) which are fraction of the genome shared IBD1, and IBD2. As <a href='http://www.biostars.org/u/3919/'>zx8754</a> mentioned above, 0.125 is third degree, 0.25 is second degree, and 0.5 is first degree, although in practice these thresholds can be too low or high due to consanguinity, and admixture.</p>
