Answers.Body: <p>Let me know if you see any potential issues with this solution. It just replaces the dots with random letters (easier/faster than aligning to reference genome).</p>
  <pre><code>def add_ref():
      # Directory path
      dir_path = '/path/'
  
      # Function to process genotypes in each row
      def process_genotypes(row):
          genotypes = np.array([x.split(&quot;:&quot;)[0] for x in row[9:]])
          if row[4] == &quot;.&quot; and not any(val in [&quot;1/1&quot;, &quot;0/1&quot;, &quot;1/0&quot;] for val in genotypes): # to make sure I'm not introducing an allele that isn't present
              # also can do if row[3] in [&quot;A&quot;, &quot;T&quot;]: then random.choice([&quot;C&quot;, &quot;G&quot;])
              if row[3] == &quot;A&quot;:
                  row[4] = &quot;C&quot;
              elif row[3] == &quot;T&quot;:
                  row[4] = &quot;G&quot;
              elif row[3] == &quot;C&quot;:
                  row[4] = random.choice([&quot;T&quot;]) # I know
              elif row[3] == &quot;G&quot;:
                  row[4] = random.choice([&quot;A&quot;])
          return row
  
      # Get all .vcf.gz files in the directory
      files = glob.glob(dir_path + '*.vcf.gz')
  
      for file in files:
          print(f&quot;Processing {file}...&quot;)
          # Name of the new file
          new_file = os.path.join(dir_path, 'new_' + os.path.basename(file))
          with gzip.open(file, 'rt') as in_f, gzip.open(new_file, 'wt') as out_f:
              for line in in_f:
                  if line.startswith('#'):
                      out_f.write(line)
                  else:
                      row = line.strip().split('\t')
                      new_row = process_genotypes(row)
                      out_f.write('\t'.join(new_row) + '\n')
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-02T10:02:51.823Z
Answers.Text: Okay I see. Python `pandas` is an alternative. If this approach runs out of memory its worth thinking about.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-02T11:25:49.523Z
Answers.Text: You best make sure to that the downstream program doesn’t use the alt allele field for something, or else it could cause problems
Answers.UserDisplayName: user438383
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T19:00:56.497Z
Answers.Text: One of the downstream programs is imputation. Not sure if it uses the alt, but I don't see why it would.
Answers.UserDisplayName: BigMistake
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-02T04:58:23.573Z
Answers.LastActivityDate: 2023-07-02T04:58:23.573Z
Answers.UserDisplayName: BigMistake
Body: <p>Here are the first few columns of my VCF v4.2 file. As you can see, the alt allele is missing. The VCF's refs and alts don't correspond to the reference genome.</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>#CHROM</th>
  <th>POS</th>
  <th>ID</th>
  <th>REF</th>
  <th>ALT</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>9</td>
  <td>77051</td>
  <td>rs9408670</td>
  <td>T</td>
  <td>C</td>
  </tr>
  <tr>
  <td>9</td>
  <td>114350</td>
  <td>rs12685480</td>
  <td>A</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>116581</td>
  <td>rs558835286</td>
  <td>G</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>117550</td>
  <td>rs267602077</td>
  <td>A</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>117720</td>
  <td>rs10959293</td>
  <td>T</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>126903</td>
  <td>rs62533602</td>
  <td>G</td>
  <td>A</td>
  </tr>
  <tr>
  <td>9</td>
  <td>133828</td>
  <td>rs78804900</td>
  <td>C</td>
  <td>T</td>
  </tr>
  <tr>
  <td>9</td>
  <td>154736</td>
  <td>rs200683662</td>
  <td>T</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>154933</td>
  <td>rs17795008</td>
  <td>T</td>
  <td>G</td>
  </tr>
  <tr>
  <td>9</td>
  <td>174425</td>
  <td>rs2998944</td>
  <td>T</td>
  <td>C</td>
  </tr>
  <tr>
  <td>9</td>
  <td>174500</td>
  <td>rs142581442</td>
  <td>G</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>175050</td>
  <td>rs62532020</td>
  <td>G</td>
  <td>A</td>
  </tr>
  <tr>
  <td>9</td>
  <td>185632</td>
  <td>rs2992854</td>
  <td>C</td>
  <td>A</td>
  </tr>
  <tr>
  <td>9</td>
  <td>188894</td>
  <td>rs3951559</td>
  <td>G</td>
  <td>A</td>
  </tr>
  <tr>
  <td>9</td>
  <td>191843</td>
  <td>rs3008128</td>
  <td>G</td>
  <td>A</td>
  </tr>
  <tr>
  <td>9</td>
  <td>202061</td>
  <td>rs117675283</td>
  <td>T</td>
  <td>.</td>
  </tr>
  <tr>
  <td>9</td>
  <td>204201</td>
  <td>rs10964134</td>
  <td>G</td>
  <td>A</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>The missing alt allele happens when the genotype is 0/0. The VCF file isn't saying anything incorrect. However, there is a tool I'm trying to use that filters out &quot;0/.&quot;genotypes.</p>
  <p>A better way of formatting them (which I'm trying to achieve) would have the ref/alt alleles correspond to the reference genome, and just have a redundant letter in the case of a homozygous genotype.</p>
  <p>I'd like my VCF to display the correct alt/ref for that position even if there are no samples with a genotype corresponding to the alt allele. How do I do this? I'm not sure what this process is called.</p>
  <p>Ideal output:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>#CHROM</th>
  <th>POS</th>
  <th>ID</th>
  <th>REF</th>
  <th>ALT</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>9</td>
  <td>77051</td>
  <td>rs9408670</td>
  <td>T</td>
  <td>C</td>
  </tr>
  <tr>
  <td>9</td>
  <td>114350</td>
  <td>rs12685480</td>
  <td>A</td>
  <td>G</td>
  </tr>
  <tr>
  <td>9</td>
  <td>116581</td>
  <td>rs558835286</td>
  <td>G</td>
  <td>A</td>
  </tr>
  </tbody>
  </table>
  </div>
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-01T22:07:29.483Z
Comments.Text: Are you sure you need to update the alt? The problem with 0/. genotypes is that period usually means “missing” so that genotype is kinda bizarre. There’s one allele we know, the reference, but also one we couldn’t figure out? If you think these should actually be homozygous reference genotypes, convert them to “0/0”.
Comments.UserDisplayName: Daniel King
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-01T22:13:23.507Z
Comments.Text: If you mean the tool can’t handle “.” in the ALT column, maybe just use awk to update all the dots to “N”? That’s a valid VCF allele but won’t be a ref.
Comments.UserDisplayName: Daniel King
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-02T04:55:11.210Z
Comments.Text: It would still filter that out, unfortunately. My solution for now is to create valid letters (ATGC) for the ref, replacing dots.
Comments.UserDisplayName: BigMistake
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-05T13:30:38.463Z
Comments.Text: You should include the genotype column of the VCF, so that we have a clearer picture. As has been mentioned, it is not the same a missing (./.) genotype than a homozigous reference. The former representation usually means the actual genotype could not be establish by the variant caller and assuming hom ref might cause mistakes in dowstream analyses a
Comments.UserDisplayName: JRodrigoF
Tags: <vcf><genome>
Title: How do I add reference/alternate alleles to my vcf file?
ThreadText: # How do I add reference/alternate alleles to my vcf file?
  
  (Asked by: username_0 on 2023-06-29)
  
  Here are the first few columns of my VCF v4.2 file. As you can see, the alt allele is missing. The VCF's refs and alts don't correspond to the reference genome.
  
  | #CHROM | POS | ID | REF | ALT |
  | --- | --- | --- | --- | --- |
  | 9 | 77051 | rs9408670 | T | C |
  | 9 | 114350 | rs12685480 | A | . |
  | 9 | 116581 | rs558835286 | G | . |
  | 9 | 117550 | rs267602077 | A | . |
  | 9 | 117720 | rs10959293 | T | . |
  | 9 | 126903 | rs62533602 | G | A |
  | 9 | 133828 | rs78804900 | C | T |
  | 9 | 154736 | rs200683662 | T | . |
  | 9 | 154933 | rs17795008 | T | G |
  | 9 | 174425 | rs2998944 | T | C |
  | 9 | 174500 | rs142581442 | G | . |
  | 9 | 175050 | rs62532020 | G | A |
  | 9 | 185632 | rs2992854 | C | A |
  | 9 | 188894 | rs3951559 | G | A |
  | 9 | 191843 | rs3008128 | G | A |
  | 9 | 202061 | rs117675283 | T | . |
  | 9 | 204201 | rs10964134 | G | A |
  
  The missing alt allele happens when the genotype is 0/0. The VCF file isn't saying anything incorrect. However, there is a tool I'm trying to use that filters out "0/."genotypes.
  
  A better way of formatting them (which I'm trying to achieve) would have the ref/alt alleles correspond to the reference genome, and just have a redundant letter in the case of a homozygous genotype.
  
  I'd like my VCF to display the correct alt/ref for that position even if there are no samples with a genotype corresponding to the alt allele. How do I do this? I'm not sure what this process is called.
  
  Ideal output:
  
  | #CHROM | POS | ID | REF | ALT |
  | --- | --- | --- | --- | --- |
  | 9 | 77051 | rs9408670 | T | C |
  | 9 | 114350 | rs12685480 | A | G |
  | 9 | 116581 | rs558835286 | G | A |
  
  **Tags:** vcf, genome
  
  > **Comments:**
  > **username_1**: Are you sure you need to update the alt? The problem with 0/. genotypes is that period usually means “missing” so that genotype is kinda bizarre. There’s one allele we know, the reference, but also one we couldn’t figure out? If you think these should actually be homozygous reference genotypes, convert them to “0/0”.  
  > **username_1**: If you mean the tool can’t handle “.” in the ALT column, maybe just use awk to update all the dots to “N”? That’s a valid VCF allele but won’t be a ref.  
  > **username_0**: It would still filter that out, unfortunately. My solution for now is to create valid letters (ATGC) for the ref, replacing dots.  
  > **username_2**: You should include the genotype column of the VCF, so that we have a clearer picture. As has been mentioned, it is not the same a missing (./.) genotype than a homozigous reference. The former representation usually means the actual genotype could not be establish by the variant caller and assuming hom ref might cause mistakes in dowstream analyses a  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2023-07-02. Score: 1
  
  Let me know if you see any potential issues with this solution. It just replaces the dots with random letters (easier/faster than aligning to reference genome).
  
  ```
  def add_ref():
      # Directory path
      dir_path = '/path/'
  
      # Function to process genotypes in each row
      def process_genotypes(row):
          genotypes = np.array([x.split(":")[0] for x in row[9:]])
          if row[4] == "." and not any(val in ["1/1", "0/1", "1/0"] for val in genotypes): # to make sure I'm not introducing an allele that isn't present
              # also can do if row[3] in ["A", "T"]: then random.choice(["C", "G"])
              if row[3] == "A":
                  row[4] = "C"
              elif row[3] == "T":
                  row[4] = "G"
              elif row[3] == "C":
                  row[4] = random.choice(["T"]) # I know
              elif row[3] == "G":
                  row[4] = random.choice(["A"])
          return row
  
      # Get all .vcf.gz files in the directory
      files = glob.glob(dir_path + '*.vcf.gz')
  
      for file in files:
          print(f"Processing {file}...")
          # Name of the new file
          new_file = os.path.join(dir_path, 'new_' + os.path.basename(file))
          with gzip.open(file, 'rt') as in_f, gzip.open(new_file, 'wt') as out_f:
              for line in in_f:
                  if line.startswith('#'):
                      out_f.write(line)
                  else:
                      row = line.strip().split('\t')
                      new_row = process_genotypes(row)
                      out_f.write('\t'.join(new_row) + '\n')
  
  ```
  
  > **Comments:**
  > **username_3**: Okay I see. Python `pandas` is an alternative. If this approach runs out of memory its worth thinking about.  
  > **username_4**: You best make sure to that the downstream program doesn’t use the alt allele field for something, or else it could cause problems  
  > **username_0**: One of the downstream programs is imputation. Not sure if it uses the alt, but I don't see why it would.  
  
  ---
  
  