Answers.Body: <p>If your file doesn't contain the rsIDs, you can use a tool like <a href="https://grch37.ensembl.org/Tools/VEP" rel="nofollow noreferrer">Ensembl's VEP</a> to annotate the file and get rsIDs as well. Of course, not all variants have an rsID so don't expect to find one for everything reported in your file.</p>
  <p>More importantly, if these are the raw, unfiltered data from commercial direct-to-consumer sequencing products like 23&amp;me, you need to be aware that they are incredibly untrustworthy. This is why 23&amp;me don't include them in their official report. They only show you the quality results that have been filtered. When working with the raw data, you are very likely to have completely bogus results and will need to have a full VCF file and filter on various quality criteria.</p>
  <p>I have looked at the raw 23&amp;me data from 4 individuals, and <em>all</em> of them had extremely rare pathogenic variants reported that would basically have killed them long before they learned to walk. They were clearly false positives.</p>
  <p>I stress that I am not suggesting these companies are producing crap: none of these variants was reported in the official report of 23&amp;me, they were filtered out. The bad calls are only present when you work on the raw, unfiltered data but if that is what you are working on, you  need to be extremely careful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-12T23:14:50.853Z
Answers.Text: Is there a downloadable file which 'maps' chromosome, position to rsid? According to this https://bioinformatics.stackexchange.com/a/14193/17155 one can download a dbSNP database file, with caveats - does this file allow you to map chr:pos to rsid? The caveats mentioned in that answer are that dbSNP build numbers are important (range mentioned is 152, 153, 154) but that reference genome is not. What build is suitable for use with SNPedia? Would a suitable file for Dante, using GRCh37, be say https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b150_GRCh37p13/VCF/00-All.vcf.gz
Answers.UserDisplayName: Pete
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-12T23:20:35.610Z
Answers.Text: Ensembl's VEP, that you mentioned, seems to return vast amounts of data, tens of rows per chr:pos, with reams of redundant data, and a limited query size, and very long waits even for tiny queries. Unless I am misunderstanding and the 'mapping' is a lot more complicated than chr:pos to rsid, it will be easier to just do it locally on my PC after a longish single download of a few GB aided by a bit of fast C code and an SSD.
Answers.UserDisplayName: Pete
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-12T23:47:05.347Z
Answers.Text: @Pete are you really only giving chr and position, with no actual variant? You said you had a VCF file, do you not?
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-13T07:30:30.460Z
Answers.Text: Sorry, you're absolutely right, I have a long line of data associated with each chr:pos - format is chr, pos, <period (in place of rsid)>, reference base, my actual variant base(s), and then there's several quality fields and many more fields. The fields are described in the file header, there must be about 30 more fields iirc (I'm not in front of the file right now). Many of these fields I don't yet understand. One thing I'm still unsure of is if you have 2 different possible alternative bases at a particular position, do they get different rsids or is it the same rsid with different 'types'
Answers.UserDisplayName: Pete
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-13T09:37:09.883Z
Answers.Text: @Pete That... isn't a VCF and isn't even similar to the image of a file you showed. That's why I asked you for an example of your file. Instead, you gave me a photograph (images of text are useless, we cannot run text parsing tools on them) and now you tell me it was a photograph that isn't even related to your data? Please don't do this. That invalidates the entire question. So please go and edit your question and include some _actual lines_ from your file and the output you expect from them. As for rsIDs, sometimes one rsID corresponds to a specific variant, other times it can be many.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-13T10:12:05.167Z
Answers.Text: No the above linked blog post is not me, but I'm the guy who contacted Shay originally about his app, and he was looking into it for me. Anyway I'll post a section of my own file in a few minutes, need to get to a pc.
Answers.UserDisplayName: Pete
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-04T04:37:34.180Z
Answers.Text: @Pete Did it take more time than expected?
Answers.UserDisplayName: Léon Pelletier
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-11T16:18:58.230Z
Answers.LastActivityDate: 2023-03-11T16:18:58.230Z
Answers.UserDisplayName: terdon
Body: <p>Is there an easy way to grab the rsids (hg38) of a VCF file? I know one of the tabs can contain RSIDs, but this file doesn't contain any.</p>
  <p>I have a software I created for mapping 23AndMe DTC testing to SNPedia, and I'm trying to integrate more DTC testing. (<a href="https://www.github.com/mentatpsi/osgenome" rel="nofollow noreferrer">https://www.github.com/mentatpsi/osgenome</a>)</p>
  <p>The user found this <a href="https://gist.github.com/KlausTrainer/26c2996cf32677e4c107bd8aaee67794" rel="nofollow noreferrer">https://gist.github.com/KlausTrainer/26c2996cf32677e4c107bd8aaee67794</a> which is a script to use variant summary data to map to the RSIDs, but I'm hesitant to perform a SQL dump to find a table that contains
  chromosome, postion, and rsID columns.</p>
  <p>As a bonus question, 23AndMe reports all positive orientations. How would I go about converting strands to match this orientation? Does the VCF contain information that would allow this action autonomously?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-11T14:05:36.360Z
Comments.Text: Can you show us an example of your file? VCFs do not need to contain rsIDs and often don't, as you see. Also, if you are referring to the raw, unfiltered data that you can download from 23&me, you need to be very, very careful with them because, at least when I looked at them last, they don't contain any quality information and will contain many, many false positives. Which is why they are not included in the official results you get on their webpage.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-11T16:03:37.323Z
Comments.Text: I do have a disclaimer on my application that Raw Data coming from DTC vendors have a false positive rate approaching 40% per a research article found in a Bioinformatics magazine.
Comments.UserDisplayName: Shay Maor
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-11T16:09:31.023Z
Comments.Text: If my understanding of the data is correct... this is the format: https://www.beholdgenealogy.com/blog/?p=2879
Comments.UserDisplayName: Shay Maor
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-11T22:22:10.983Z
Comments.Text: Re the false positives - with this VCF data, there appears to be several fields of quality data. How useful I don't know, being a beginner to this subject. I'll post a  the header and a few lines of the VCF later in a reply. But if you have, for instance, the number of reads (is that depth? Still getting the terminology) that match this particular variant, I assume that would allow you to provide a confidence level.
Comments.UserDisplayName: Pete
Tags: <vcf><gene><snp><genotyping><rsid>
Title: Converting a Dante Lab VCF file
ThreadText: # Converting a Dante Lab VCF file
  
  (Asked by: username_0 on 2023-03-11)
  
  Is there an easy way to grab the rsids (hg38) of a VCF file? I know one of the tabs can contain RSIDs, but this file doesn't contain any.
  
  I have a software I created for mapping 23AndMe DTC testing to SNPedia, and I'm trying to integrate more DTC testing. (<https://www.github.com/mentatpsi/osgenome>)
  
  The user found this <https://gist.github.com/KlausTrainer/26c2996cf32677e4c107bd8aaee67794> which is a script to use variant summary data to map to the RSIDs, but I'm hesitant to perform a SQL dump to find a table that contains
  chromosome, postion, and rsID columns.
  
  As a bonus question, 23AndMe reports all positive orientations. How would I go about converting strands to match this orientation? Does the VCF contain information that would allow this action autonomously?
  
  **Tags:** vcf, gene, snp, genotyping, rsid
  
  > **Comments:**
  > **username_1**: Can you show us an example of your file? VCFs do not need to contain rsIDs and often don't, as you see. Also, if you are referring to the raw, unfiltered data that you can download from 23&me, you need to be very, very careful with them because, at least when I looked at them last, they don't contain any quality information and will contain many, many false positives. Which is why they are not included in the official results you get on their webpage.  
  > **username_0**: I do have a disclaimer on my application that Raw Data coming from DTC vendors have a false positive rate approaching 40% per a research article found in a Bioinformatics magazine.  
  > **username_0**: If my understanding of the data is correct... this is the format: https://www.beholdgenealogy.com/blog/?p=2879  
  > **username_2**: Re the false positives - with this VCF data, there appears to be several fields of quality data. How useful I don't know, being a beginner to this subject. I'll post a  the header and a few lines of the VCF later in a reply. But if you have, for instance, the number of reads (is that depth? Still getting the terminology) that match this particular variant, I assume that would allow you to provide a confidence level.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-11. Score: 1 | Accepted answer
  
  If your file doesn't contain the rsIDs, you can use a tool like [Ensembl's VEP](https://grch37.ensembl.org/Tools/VEP) to annotate the file and get rsIDs as well. Of course, not all variants have an rsID so don't expect to find one for everything reported in your file.
  
  More importantly, if these are the raw, unfiltered data from commercial direct-to-consumer sequencing products like 23&me, you need to be aware that they are incredibly untrustworthy. This is why 23&me don't include them in their official report. They only show you the quality results that have been filtered. When working with the raw data, you are very likely to have completely bogus results and will need to have a full VCF file and filter on various quality criteria.
  
  I have looked at the raw 23&me data from 4 individuals, and *all* of them had extremely rare pathogenic variants reported that would basically have killed them long before they learned to walk. They were clearly false positives.
  
  I stress that I am not suggesting these companies are producing crap: none of these variants was reported in the official report of 23&me, they were filtered out. The bad calls are only present when you work on the raw, unfiltered data but if that is what you are working on, you need to be extremely careful.
  
  > **Comments:**
  > **username_2**: Is there a downloadable file which 'maps' chromosome, position to rsid? According to this https://bioinformatics.stackexchange.com/a/14193/17155 one can download a dbSNP database file, with caveats - does this file allow you to map chr:pos to rsid? The caveats mentioned in that answer are that dbSNP build numbers are important (range mentioned is 152, 153, 154) but that reference genome is not. What build is suitable for use with SNPedia? Would a suitable file for Dante, using GRCh37, be say https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b150_GRCh37p13/VCF/00-All.vcf.gz  
  > **username_2**: Ensembl's VEP, that you mentioned, seems to return vast amounts of data, tens of rows per chr:pos, with reams of redundant data, and a limited query size, and very long waits even for tiny queries. Unless I am misunderstanding and the 'mapping' is a lot more complicated than chr:pos to rsid, it will be easier to just do it locally on my PC after a longish single download of a few GB aided by a bit of fast C code and an SSD.  
  > **username_1**: @username_2 are you really only giving chr and position, with no actual variant? You said you had a VCF file, do you not?  
  > **username_2**: Sorry, you're absolutely right, I have a long line of data associated with each chr:pos - format is chr, pos, <period (in place of rsid)>, reference base, my actual variant base(s), and then there's several quality fields and many more fields. The fields are described in the file header, there must be about 30 more fields iirc (I'm not in front of the file right now). Many of these fields I don't yet understand. One thing I'm still unsure of is if you have 2 different possible alternative bases at a particular position, do they get different rsids or is it the same rsid with different 'types'  
  > **username_1**: @username_2 That... isn't a VCF and isn't even similar to the image of a file you showed. That's why I asked you for an example of your file. Instead, you gave me a photograph (images of text are useless, we cannot run text parsing tools on them) and now you tell me it was a photograph that isn't even related to your data? Please don't do this. That invalidates the entire question. So please go and edit your question and include some _actual lines_ from your file and the output you expect from them. As for rsIDs, sometimes one rsID corresponds to a specific variant, other times it can be many.  
  > **username_2**: No the above linked blog post is not me, but I'm the guy who contacted username_0 originally about his app, and he was looking into it for me. Anyway I'll post a section of my own file in a few minutes, need to get to a pc.  
  > **username_3**: @username_2 Did it take more time than expected?  
  
  ---
  
  