Answers.Body: <p>It is. A VCF ought to be sub-settable to a specific list of samples. The only thing you have to come up with for your specific case, is a list of 1K random sample IDs within the original VCF.</p>
  <p>Geenerally speaking, knowing that you can do this and how involves getting familiar with the <strong><a href="https://samtools.github.io/hts-specs/VCFv4.2.pdf" rel="nofollow noreferrer">VCF specification</a></strong> as well as the main bioinfo tool used to manipulate and work with this format: <strong><a href="https://samtools.github.io/bcftools/bcftools.html" rel="nofollow noreferrer">BCFtools</a></strong></p>
  <p>Let's use <strong><a href="https://www.internationalgenome.org/data/" rel="nofollow noreferrer">1000 genomes data</a></strong> as an example.</p>
  <p>You can use a bit of UNIX together with BCFtools in order to get a list of 1K random IDs from individuals:</p>
  <p>(sort -R instead of shuf might work too)</p>
  <pre><code>bcftools query -l ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz | shuf | head -1000 &gt; myRandomIDs.txt
  </code></pre>
  <p>Then again BCFtools to subset the list of individuals from the original VCF and into a new VCF file:</p>
  <pre><code>bcftools view --samples-file myRandomIDs.txt ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz -o myNewVCF.vcf
  </code></pre>
  <p>If you dig a bit deeper into the BCFtools documentation, you will find many things you can do extra if you wanted, like re-calculating the allele frequencies based on this new cohort of individuals, if you wanted that too. Or subset for positions/variants, instead of individuals.</p>
  <p><strong>GnomAD data</strong></p>
  <p>Using GnomAD data, selecting 1K random samples is impossible since data is aggregated and instead of genotypes per individual (and individuals you can select thus) we only have the frequency of the variants across various sub-cohorts within the GnomAD project. No individual level data.</p>
  <p>The most you can do is select one of such sub-cohorts and filter for variants with a frequency &gt; 0 (or a range) in such cohort. You could get variants present only and only is a specific cohort too, by using either frequency information or other info such as allele counts, etc.</p>
  <p>The example below filters for variants with a frequency above 0 in the South Asian cohort and 0 frequency in the Southern European cohort. You can have a look at the header of the VCF in order to get to know all cohorts included and the meaning of all the information contained within the INFO filed in the VCF</p>
  <pre><code>bcftools filter -i &quot;AF_sas &gt; 0 &amp; AF_nfe_seu = 0&quot; gnomad.exomes.r2.1.1.sites.21.vcf.bgz
  </code></pre>
  <p>But I'm afraid that's not exactly what you wanted to do in the first place.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-25T16:32:29.753Z
Answers.Text: The problemz is, while this works with the dataset you've given as the example, when I execute the `bcftools query -l gnomad.genomes.r2.1.sites.chr21.vcf.gz` it does not return any output. Does this indicate that in this VCF file does not include any sample ID information?
Answers.UserDisplayName: Arda Cüce
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-26T07:42:53.653Z
Answers.Text: @ArdaCüce gnomad doesn’t actually contain individual level samples, it is a vcf of aggregated samples which gives you the allele frequencies (I.e. it contains averages), so it contains no sample level information. You might want to try a different dataset which does have individual level data, depending on the question you want to ask.
Answers.UserDisplayName: user438383
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-26T09:37:23.563Z
Answers.Text: OK. I see, I for a moment thought your GnomAD VCF had sample-level data, but this makes sense actually. Please see if my updated answer provides a better answer,
Answers.UserDisplayName: JRodrigoF
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-25T13:54:55.187Z
Answers.LastActivityDate: 2022-10-26T09:35:57.703Z
Answers.LastEditDate: 2022-10-26T09:35:57.703Z
Answers.UserDisplayName: JRodrigoF
Body: <p>GnomAD has publicly available VCF files that have variant data gathered from over 15k individuals. For a university project, I want to randomly select 1k of these individuals and get the variant data belgoning to them only.</p>
  <p>I am very new to bioinformatics in general and upon checking VCF files and how they work, my current conclusion is that this task is not achievable through filtering the vcf file in any way since variants in these files have no direct connection to the genomes they were found in (no array of individual IDs in a variant entry that show which individuals had this variant).</p>
  <p>But this task was given to me by a professor whom I believe is competent to realize the task was impossible if it is like I thought, so is there something I am missing?</p>
  <p>How can I go about randomly selecting a 1000 individuals from this 15 000 pool, and then generating a new VCF file with the data from these selected 1000?</p>
  <p>Are there any fields that show which individuals genome samples showed a spesific variant entry in the VCF file?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-24T18:02:30.630Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-24T18:09:07.453Z
Comments.Text: When you say you want "1k of these genomes", I think you mean "you want the variants identified in 1k of these samples" and not the actual genomes. Is that correct? And what do you mean by "variants in these files have no direct connection to the genomes"? If this is a multi-sample VCF file, the zygosity field will tell you if this variant was found in a given sample.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-25T07:32:50.793Z
Comments.Text: Do you want to simply randomly sample 1K individuals from the VCF and produce a second VCF with the variant data of those 1K individuals ? That's what I understand,
Comments.UserDisplayName: JRodrigoF
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-25T10:17:07.573Z
Comments.Text: @JRodrigoF Yes that is exactly what I want, it seems to me that I understood some spesifications of the vcf file wrong. Is this task doable, how would one go about achieving this?
Comments.UserDisplayName: Arda Cüce
Tags: <vcf><gnomad>
Title: Is picking variants belonging to a specific group of genomes from VCF file possible?
ThreadText: # Is picking variants belonging to a specific group of genomes from VCF file possible?
  
  (Asked by: username_0 on 2022-10-24)
  
  GnomAD has publicly available VCF files that have variant data gathered from over 15k individuals. For a university project, I want to randomly select 1k of these individuals and get the variant data belgoning to them only.
  
  I am very new to bioinformatics in general and upon checking VCF files and how they work, my current conclusion is that this task is not achievable through filtering the vcf file in any way since variants in these files have no direct connection to the genomes they were found in (no array of individual IDs in a variant entry that show which individuals had this variant).
  
  But this task was given to me by a professor whom I believe is competent to realize the task was impossible if it is like I thought, so is there something I am missing?
  
  How can I go about randomly selecting a 1000 individuals from this 15 000 pool, and then generating a new VCF file with the data from these selected 1000?
  
  Are there any fields that show which individuals genome samples showed a spesific variant entry in the VCF file?
  
  **Tags:** vcf, gnomad
  
  > **Comments:**
  > **username_1**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  > **username_2**: When you say you want "1k of these genomes", I think you mean "you want the variants identified in 1k of these samples" and not the actual genomes. Is that correct? And what do you mean by "variants in these files have no direct connection to the genomes"? If this is a multi-sample VCF file, the zygosity field will tell you if this variant was found in a given sample.  
  > **username_3**: Do you want to simply randomly sample 1K individuals from the VCF and produce a second VCF with the variant data of those 1K individuals ? That's what I understand,  
  > **username_0**: @username_3 Yes that is exactly what I want, it seems to me that I understood some spesifications of the vcf file wrong. Is this task doable, how would one go about achieving this?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-10-25. Score: 0 | Accepted answer
  
  It is. A VCF ought to be sub-settable to a specific list of samples. The only thing you have to come up with for your specific case, is a list of 1K random sample IDs within the original VCF.
  
  Geenerally speaking, knowing that you can do this and how involves getting familiar with the **[VCF specification](https://samtools.github.io/hts-specs/VCFv4.2.pdf)** as well as the main bioinfo tool used to manipulate and work with this format: **[BCFtools](https://samtools.github.io/bcftools/bcftools.html)**
  
  Let's use **[1000 genomes data](https://www.internationalgenome.org/data/)** as an example.
  
  You can use a bit of UNIX together with BCFtools in order to get a list of 1K random IDs from individuals:
  
  (sort -R instead of shuf might work too)
  
  ```
  bcftools query -l ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz | shuf | head -1000 > myRandomIDs.txt
  
  ```
  
  Then again BCFtools to subset the list of individuals from the original VCF and into a new VCF file:
  
  ```
  bcftools view --samples-file myRandomIDs.txt ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz -o myNewVCF.vcf
  
  ```
  
  If you dig a bit deeper into the BCFtools documentation, you will find many things you can do extra if you wanted, like re-calculating the allele frequencies based on this new cohort of individuals, if you wanted that too. Or subset for positions/variants, instead of individuals.
  
  **GnomAD data**
  
  Using GnomAD data, selecting 1K random samples is impossible since data is aggregated and instead of genotypes per individual (and individuals you can select thus) we only have the frequency of the variants across various sub-cohorts within the GnomAD project. No individual level data.
  
  The most you can do is select one of such sub-cohorts and filter for variants with a frequency > 0 (or a range) in such cohort. You could get variants present only and only is a specific cohort too, by using either frequency information or other info such as allele counts, etc.
  
  The example below filters for variants with a frequency above 0 in the South Asian cohort and 0 frequency in the Southern European cohort. You can have a look at the header of the VCF in order to get to know all cohorts included and the meaning of all the information contained within the INFO filed in the VCF
  
  ```
  bcftools filter -i "AF_sas > 0 & AF_nfe_seu = 0" gnomad.exomes.r2.1.1.sites.21.vcf.bgz
  
  ```
  
  But I'm afraid that's not exactly what you wanted to do in the first place.
  
  > **Comments:**
  > **username_0**: The problemz is, while this works with the dataset you've given as the example, when I execute the `bcftools query -l gnomad.genomes.r2.1.sites.chr21.vcf.gz` it does not return any output. Does this indicate that in this VCF file does not include any sample ID information?  
  > **username_4**: @username_0 gnomad doesn’t actually contain individual level samples, it is a vcf of aggregated samples which gives you the allele frequencies (I.e. it contains averages), so it contains no sample level information. You might want to try a different dataset which does have individual level data, depending on the question you want to ask.  
  > **username_3**: OK. I see, I for a moment thought your GnomAD VCF had sample-level data, but this makes sense actually. Please see if my updated answer provides a better answer,  
  
  ---
  
  