Answers.Body: <p>No. The only point the specification makes is that:</p>
  
  <blockquote>
    <p>It is permitted to have multiple records with the same POS</p>
  </blockquote>
  
  <p>I suspect this is <em>intended</em> to let the file list different variants occurring at the same position, but this incidentally allows duplicate variants.</p>
  
  <p>This said, you probably don't <em>want</em> duplicate variants, because it will mess with any statistics you apply over the variants, and will unnecessarily clutter the file and slow down processing, if this happens a lot. You might also get variant annotations on one variant but not another, which can become a problem depending on how you query your variants.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-10T00:39:43.967Z
Answers.Text: Duplicate variants also point to an issue with upstream analysis. If the variants have different statistics (e.g. in this example one has qual 0.00, and the other has qual "."), then there is a bias in the counting. If the variants have the *same* statistics, then the lines are redundant.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-05T04:19:33.577Z
Answers.LastActivityDate: 2019-06-05T04:19:33.577Z
Answers.UserDisplayName: Migwell
Body: <p>I have been trying to understand an error that the <a href="https://github.com/EBIvariation/vcf-validator" rel="nofollow noreferrer">EBI's vcf_validator</a> gives when run on my vcf file. Consider this minimal file:</p>
  
  <pre><code>##fileformat=VCFv4.1
  ##reference=foo
  ##FORMAT=&lt;ID=GT,Number=1,Type=String,Description="Genotype"&gt;
  ##contig=&lt;ID=chr12&gt;
  #CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1
  chr12   529514  .   AACAC   AATAC   0.00    .   .   GT  0/1
  chr12   529516  .   C   T   .   PASS    .   GT  0/1
  </code></pre>
  
  <p>If I then run vcf_validator on it, it complains about:</p>
  
  <pre><code><span class="math-container">$ cat $</span>(LC_ALL="C" ~/vcf_validator_linux  -i test.vcf |&amp; awk '/written to/{print $NF}' )
  According to the VCF specification, the input file is not valid
  Error: Duplicated variant chr12:529516:C&gt;T found. This occurs 2 time(s), first time in line 6.
  </code></pre>
  
  <p>This is because the two variants are indeed the same. Changing 
  <code>AACAC</code> to <code>AATAC</code> at position 529514 just means change <code>C</code> to <code>T</code> at position 529516.</p>
  
  <p>OK, but how is that against the VCF standard? What makes that an invalid VCF file? I am trying to automatically validate input VCF files and stop the analysis if they're invalid, but this seems like a perfectly valid file. Why is  <code>vcf_validator</code> not letting it through? The <code>vcf-validator</code> perl script from vcftools doesn't complain about this. So is it just a bug in  EBI's <code>vcf_validator</code> or is it really somehow illegal to have duplicate variants in a VCF file according to <a href="https://samtools.github.io/hts-specs/VCFv4.3.pdf" rel="nofollow noreferrer">the specs</a>?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-05T09:30:15.313Z
Comments.Text: What is the use case for having duplicate variant records in a VCF file?
Comments.UserDisplayName: John Marshall
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-06T13:22:03.337Z
Comments.Text: @JohnMarshall none that I know of. But someone sent me a file with such duplicates and I was surprised that they caused the `vcf_validator` to consider the file invalid, so I asked if there was anything in the specs that would actually make such files invalid.
Comments.UserDisplayName: terdon
Tags: <vcf><variation><vcftools>
Title: Are duplicate variants against the VCF standard?
ThreadText: # Are duplicate variants against the VCF standard?
  
  (Asked by: username_0 on 2019-02-23)
  
  I have been trying to understand an error that the [EBI's vcf\_validator](https://github.com/EBIvariation/vcf-validator) gives when run on my vcf file. Consider this minimal file:
  
  ```
  ##fileformat=VCFv4.1
  ##reference=foo
  ##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
  ##contig=<ID=chr12>
  #CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1
  chr12   529514  .   AACAC   AATAC   0.00    .   .   GT  0/1
  chr12   529516  .   C   T   .   PASS    .   GT  0/1
  
  ```
  
  If I then run vcf\_validator on it, it complains about:
  
  ```
  $ cat $(LC_ALL="C" ~/vcf_validator_linux  -i test.vcf |& awk '/written to/{print $NF}' )
  According to the VCF specification, the input file is not valid
  Error: Duplicated variant chr12:529516:C>T found. This occurs 2 time(s), first time in line 6.
  
  ```
  
  This is because the two variants are indeed the same. Changing
  `AACAC` to `AATAC` at position 529514 just means change `C` to `T` at position 529516.
  
  OK, but how is that against the VCF standard? What makes that an invalid VCF file? I am trying to automatically validate input VCF files and stop the analysis if they're invalid, but this seems like a perfectly valid file. Why is `vcf_validator` not letting it through? The `vcf-validator` perl script from vcftools doesn't complain about this. So is it just a bug in EBI's `vcf_validator` or is it really somehow illegal to have duplicate variants in a VCF file according to [the specs](https://samtools.github.io/hts-specs/VCFv4.3.pdf)?
  
  **Tags:** vcf, variation, vcftools
  
  > **Comments:**
  > **username_1**: What is the use case for having duplicate variant records in a VCF file?  
  > **username_0**: @username_1 none that I know of. But someone sent me a file with such duplicates and I was surprised that they caused the `vcf_validator` to consider the file invalid, so I asked if there was anything in the specs that would actually make such files invalid.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-06-05. Score: 8 | Accepted answer
  
  No. The only point the specification makes is that:
  
  > It is permitted to have multiple records with the same POS
  
  I suspect this is *intended* to let the file list different variants occurring at the same position, but this incidentally allows duplicate variants.
  
  This said, you probably don't *want* duplicate variants, because it will mess with any statistics you apply over the variants, and will unnecessarily clutter the file and slow down processing, if this happens a lot. You might also get variant annotations on one variant but not another, which can become a problem depending on how you query your variants.
  
  > **Comments:**
  > **username_3**: Duplicate variants also point to an issue with upstream analysis. If the variants have different statistics (e.g. in this example one has qual 0.00, and the other has qual "."), then there is a bias in the counting. If the variants have the *same* statistics, then the lines are redundant.  
  
  ---
  
  