Answers.Body: <p>using <strong><a href="http://lindenb.github.io/jvarkit/VcfFilterJdk.html" rel="nofollow noreferrer">vcffilterjdk</a></strong>:</p>
  
  <p>you want to keep the variant if there is no INFO/CAF.</p>
  
  <p>Otherwise, get All values of CAF <a href="https://samtools.github.io/htsjdk/javadoc/htsjdk/htsjdk/variant/variantcontext/VariantContext.html#getAttributeAsStringList-java.lang.String-java.lang.String-" rel="nofollow noreferrer">as a String list</a> (because it is declared as a string in the VCF header...), stream those data, skip the first value for REF , convert each string to double, you want to keep the variant if any allele is rare (&lt; 0.01)</p>
  
  <pre><code>java -jar dist/vcffilterjdk.jar -e 'return !variant.hasAttribute("CAF") || variant.variant.getAttributeAsStringList("CAF","1.0").stream().skip(1L).map(A-&gt;Double.parseDouble(A)).anyMatch(A-&gt;A&lt;0.01);' input.vcf
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-22T21:55:49.883Z
Answers.Text: I have not encountered this tool before, but in my experience, if Pierre suggests something it will work as advertised.  I was going to suggest a tool that I’d read about but never got around until last week: bioawk.  It was blazingly fast.
Answers.UserDisplayName: mdperry
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-03T09:07:10.373Z
Answers.LastActivityDate: 2018-07-03T15:09:56.353Z
Answers.LastEditDate: 2018-07-03T15:09:56.353Z
Answers.UserDisplayName: Pierre
Answers.Body: <p>This could be done with <code>bcftools view</code>. To include variants where the first element in CAF is > <em>x</em>:</p>
  
  <pre><code>bcftools view -i 'INFO/CAF[0] &gt; 0.1' ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz
  </code></pre>
  
  <p>However, it appears that CAF is encoded as a list of strings in this input so the <code>&gt;</code> operator cannot be used. In addition, you may want more control on the filtering. In this case, I would suggest using a dedicated library in your favourite language for dealing with VCF files. In python, you could use pysam along these lines:</p>
  
  <pre><code>python -c "import pysam
  invcf= pysam.VariantFile('ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz')
  
  # Write to stdout
  outvcf= pysam.VariantFile('-', mode= 'w', header= invcf.header)
  
  for line in invcf:
      if float(line.info['CAF'][0]) &gt; 0.1:
          outvcf.write(line)
  outvcf.close()
  invcf.close()
  "
  </code></pre>
  
  <p>I would stay away from parsing VCF files with regex, awk, or custom text processing scripts as even if the format is quite well defined they can be quite tricky.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-02T13:52:16.200Z
Answers.Text: The first value is for the reference allele. Presumably, the OP is interested in filtering by the variant's frequency instead. So it would have to be based on elements 1 through the end of the array.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-02T13:35:04.833Z
Answers.LastActivityDate: 2018-06-02T13:35:04.833Z
Answers.UserDisplayName: dariober
Body: <p>In my gzipped VCF file I have many variants, and they report allele frequency such as:</p>
  
  <pre><code>##INFO=&lt;ID=CAF,Number=.,Type=String,Description="An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter"&gt;
  </code></pre>
  
  <p>and </p>
  
  <pre><code>CAF=0.5747,0.4253
  </code></pre>
  
  <p>I would like to filter out the variants >= 1% allelle frequency. How do I do that? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-01T07:05:35.407Z
Comments.Text: Did you try any program or regex? How big is your VCF file ? (perhaps you could post some lines with  what you want and what you don't want) to test it
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-01T07:06:55.263Z
Comments.Text: @Llopis I want all lines that the allele frequency of the alternative allele is >= 0.01. In my example, I'd keep it because 0.4253 > 0.01.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-01T07:07:33.433Z
Comments.Text: @Llopis My VCF files are big and gzipped. Direct regression expression won't work.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-01T07:54:20.760Z
Comments.Text: I agree with @Llopis, you should post a few lines of your vcf file to give a workable example. You should probably use AWK.
Comments.UserDisplayName: benn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-01T08:36:34.627Z
Comments.Text: How do you want to deal with multi-allelic variants (e.g. `A C,T`) which will have >2 CAF values? Do you want _all_ of them to be <1% or _any_? And as others have said, we can't really do this without a few example lines to work with. Ideally, give us a few lines including a multiallelic one and show us your desired output.
Comments.UserDisplayName: terdon
Tags: <vcf><file-formats>
Title: How to filter for "CAF=0.5747,0.4253"
ThreadText: # How to filter for "CAF=0.5747,0.4253"
  
  (Asked by: username_0 on 2018-06-01)
  
  In my gzipped VCF file I have many variants, and they report allele frequency such as:
  
  ```
  ##INFO=<ID=CAF,Number=.,Type=String,Description="An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter">
  
  ```
  
  and
  
  ```
  CAF=0.5747,0.4253
  
  ```
  
  I would like to filter out the variants >= 1% allelle frequency. How do I do that?
  
  **Tags:** vcf, file-formats
  
  > **Comments:**
  > **username_1**: Did you try any program or regex? How big is your VCF file ? (perhaps you could post some lines with  what you want and what you don't want) to test it  
  > **username_0**: @Llopis I want all lines that the allele frequency of the alternative allele is >= 0.01. In my example, I'd keep it because 0.4253 > 0.01.  
  > **username_0**: @Llopis My VCF files are big and gzipped. Direct regression expression won't work.  
  > **username_2**: I agree with @Llopis, you should post a few lines of your vcf file to give a workable example. You should probably use AWK.  
  > **username_3**: How do you want to deal with multi-allelic variants (e.g. `A C,T`) which will have >2 CAF values? Do you want _all_ of them to be <1% or _any_? And as others have said, we can't really do this without a few example lines to work with. Ideally, give us a few lines including a multiallelic one and show us your desired output.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2018-07-03. Score: 4
  
  using **[vcffilterjdk](http://lindenb.github.io/jvarkit/VcfFilterJdk.html)**:
  
  you want to keep the variant if there is no INFO/CAF.
  
  Otherwise, get All values of CAF [as a String list](https://samtools.github.io/htsjdk/javadoc/htsjdk/htsjdk/variant/variantcontext/VariantContext.html#getAttributeAsStringList-java.lang.String-java.lang.String-) (because it is declared as a string in the VCF header...), stream those data, skip the first value for REF , convert each string to double, you want to keep the variant if any allele is rare (< 0.01)
  
  ```
  java -jar dist/vcffilterjdk.jar -e 'return !variant.hasAttribute("CAF") || variant.variant.getAttributeAsStringList("CAF","1.0").stream().skip(1L).map(A->Double.parseDouble(A)).anyMatch(A->A<0.01);' input.vcf
  
  ```
  
  > **Comments:**
  > **username_5**: I have not encountered this tool before, but in my experience, if username_4 suggests something it will work as advertised.  I was going to suggest a tool that I’d read about but never got around until last week: bioawk.  It was blazingly fast.  
  
  ---
  
  ### Answer by username_6 on 2018-06-02. Score: 0
  
  This could be done with `bcftools view`. To include variants where the first element in CAF is > *x*:
  
  ```
  bcftools view -i 'INFO/CAF[0] > 0.1' ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz
  
  ```
  
  However, it appears that CAF is encoded as a list of strings in this input so the `>` operator cannot be used. In addition, you may want more control on the filtering. In this case, I would suggest using a dedicated library in your favourite language for dealing with VCF files. In python, you could use pysam along these lines:
  
  ```
  python -c "import pysam
  invcf= pysam.VariantFile('ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz')
  
  # Write to stdout
  outvcf= pysam.VariantFile('-', mode= 'w', header= invcf.header)
  
  for line in invcf:
      if float(line.info['CAF'][0]) > 0.1:
          outvcf.write(line)
  outvcf.close()
  invcf.close()
  "
  
  ```
  
  I would stay away from parsing VCF files with regex, awk, or custom text processing scripts as even if the format is quite well defined they can be quite tricky.
  
  > **Comments:**
  > **username_3**: The first value is for the reference allele. Presumably, the OP is interested in filtering by the variant's frequency instead. So it would have to be based on elements 1 through the end of the array.  
  
  ---
  
  