Answers.Body: <p>Coding: first and foremost all operations should be performed directly in <code>pandas</code>. It is much more efficient and alot easier. You don't need all the loops, <code>pandas</code> does that. <code>apply</code> is okay. I would use a very different strategy (below).</p>
  <hr />
  <p>The actually contents of the <code>csv</code> are not clear.</p>
  <p>This is what I think might have happened: there's   multiple VCFs against the same reference I assume these were loaded and a database <code>join</code> operation performed using the reference genome column 'REF' as the index for the <code>join</code> across all input VCFs. I've placed this here because I'm not 100% sure what the input file represents - if it's not what I think it is - thats how I'd get to the allelic frequency.</p>
  <p>Why do I think that?</p>
  <ol>
  <li>Without multiple strains the allelic frequency cannot be calculated.</li>
  <li>The file states '200 samples'</li>
  <li>The presence of multiple alleles in the 'ALT' column.</li>
  </ol>
  <p>The problems are:</p>
  <ul>
  <li>heterozygosity. This assumes falciparum is in its haploid state (it has a diploid stage). If its the diploid stage the allelic frequency cannot be calculated using this input file because the linkage was discarded (which is really important)</li>
  <li>The underlying problem is the number of actual strains isn't explicitly stated .</li>
  <li>the frequency of the 'ALT' doesn't appear to be declared</li>
  <li>even if the a frequency was declared (but not presented) that must be linked to the 'ALT' column, each allele needs a exact frequency, otherwise population genetics will not work on this data.</li>
  </ul>
  <p><strong>First conclusion</strong> The allelic frequency is performed across each row using the reference genome as the reference via <code>value_counts()</code> divided by the number of strains and presented in its own column. Somehow the relevant columns are missing - will need to see them.</p>
  <p><strong>However</strong> Again on the information presented I don't think it can be done because if this represents 200 samples the frequency of a given allele is not declared in this data, or simply the number of 'ALT's across all VCFs. Without accessing that number the allelic frequency is not doable because it's entirely dependent on it.</p>
  <p><strong>Possible alternative conclusion</strong> Its possible the question needs to be redefined not as <em>allelic frequency</em> - this has an exact definition within population genetics, but <em>genetic diversity</em>, i.e. remove the term 'frequency' altogether. What I think is being calculated is genetic diversity of all samples in the file against a reference genome. If that is correct it would be good to change the question accordingly. These are different mathematical entities.</p>
  <p>Whatever operation is required is easy to calculate, but its obtaining that information from the input file and the precise nature of the investigation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-11T22:58:06.740Z
Answers.LastActivityDate: 2023-04-12T00:09:29.843Z
Answers.LastEditDate: 2023-04-12T00:09:29.843Z
Answers.UserDisplayName: M__
Body: <p>Organism under investigation is  <em>Plasmodium falciparum</em>. How to calculate the allele frequency for each row?</p>
  <p><a href="https://i.sstatic.net/Q6EOC.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Q6EOC.jpg" alt="enter image description here" /></a></p>
  <p>I tried with this code:</p>
  <pre><code>    import pandas as pd
  
  # Read CSV file
  # df = pd.read_csv(&quot;Pf_M76611.pf7.200_samples - Copy.csv&quot;)
  
  df= pd.read_csv(&quot;/content/Pf_M76611.pf7.200_samples - Copy.csv&quot;)
  
  # Define function to calculate allele frequencies
  def calc_allele_freq(row):
      # Get total number of samples
      num_samples = len(row) - 9
  
      # Get reference allele
      ref_allele = row['REF']
  
      # Get alternative alleles
      alt_alleles = row['ALT'].split(',')
  
      # Initialize list to store allele frequencies
      allele_freqs = []
  
      # Loop over alternative alleles
      for alt_allele in alt_alleles:
          # Initialize count for this allele
          count = 0
  
          # Loop over samples
          for i in range(9, len(row)):
              # Get genotype for this sample
              gt = row[i].split(':')[0]
  
              # Count this allele if it is present in the genotype
              if alt_allele in gt:
                  count += 1
  
          # Calculate frequency for this allele
          freq = count / num_samples
  
          # Add frequency to list
          allele_freqs.append(freq)
  
      # Add frequency for reference allele
      allele_freqs.insert(0, 1 - sum(allele_freqs))
  
      # Return allele list and frequency list as a tuple
      allele_list = [ref_allele] + alt_alleles
      freq_list = [round(freq, 1) for freq in allele_freqs]
      return (allele_list, freq_list)
  
  # Apply function to each row of the dataframe
  allele_freq_df = df.apply(calc_allele_freq, axis=1, result_type='expand')
  
  # Add columns to dataframe
  df['Allele List'] = allele_freq_df[0]
  df['Proportion List'] = allele_freq_df[1]
  
  # Select columns for output and drop duplicates
  # output_df = df[['#CHROM', 'POS', 'Allele List', 'Proportion List']].drop_duplicates()
  
  # # Convert output dataframe to string and print
  # print(output_df.to_string(index=False))
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-11T20:58:26.960Z
Comments.Text: Could you please show that table as text format? It's quite difficult to read on my mobile device.
Comments.UserDisplayName: gringer
Tags: <python><sequence-analysis><genomics><allele-frequency>
Title: How to calculate Allele frequency from vcf file
ThreadText: # How to calculate Allele frequency from vcf file
  
  (Asked by: username_0 on 2023-04-10)
  
  Organism under investigation is *Plasmodium falciparum*. How to calculate the allele frequency for each row?
  
  [![enter image description here](https://i.sstatic.net/Q6EOC.jpg)](https://i.sstatic.net/Q6EOC.jpg)
  
  I tried with this code:
  
  ```
      import pandas as pd
  
  # Read CSV file
  # df = pd.read_csv("Pf_M76611.pf7.200_samples - Copy.csv")
  
  df= pd.read_csv("/content/Pf_M76611.pf7.200_samples - Copy.csv")
  
  # Define function to calculate allele frequencies
  def calc_allele_freq(row):
      # Get total number of samples
      num_samples = len(row) - 9
  
      # Get reference allele
      ref_allele = row['REF']
  
      # Get alternative alleles
      alt_alleles = row['ALT'].split(',')
  
      # Initialize list to store allele frequencies
      allele_freqs = []
  
      # Loop over alternative alleles
      for alt_allele in alt_alleles:
          # Initialize count for this allele
          count = 0
  
          # Loop over samples
          for i in range(9, len(row)):
              # Get genotype for this sample
              gt = row[i].split(':')[0]
  
              # Count this allele if it is present in the genotype
              if alt_allele in gt:
                  count += 1
  
          # Calculate frequency for this allele
          freq = count / num_samples
  
          # Add frequency to list
          allele_freqs.append(freq)
  
      # Add frequency for reference allele
      allele_freqs.insert(0, 1 - sum(allele_freqs))
  
      # Return allele list and frequency list as a tuple
      allele_list = [ref_allele] + alt_alleles
      freq_list = [round(freq, 1) for freq in allele_freqs]
      return (allele_list, freq_list)
  
  # Apply function to each row of the dataframe
  allele_freq_df = df.apply(calc_allele_freq, axis=1, result_type='expand')
  
  # Add columns to dataframe
  df['Allele List'] = allele_freq_df[0]
  df['Proportion List'] = allele_freq_df[1]
  
  # Select columns for output and drop duplicates
  # output_df = df[['#CHROM', 'POS', 'Allele List', 'Proportion List']].drop_duplicates()
  
  # # Convert output dataframe to string and print
  # print(output_df.to_string(index=False))
  
  ```
  
  **Tags:** python, sequence-analysis, genomics, allele-frequency
  
  > **Comments:**
  > **username_1**: Could you please show that table as text format? It's quite difficult to read on my mobile device.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-11. Score: 0
  
  Coding: first and foremost all operations should be performed directly in `pandas`. It is much more efficient and alot easier. You don't need all the loops, `pandas` does that. `apply` is okay. I would use a very different strategy (below).
  
  ---
  
  The actually contents of the `csv` are not clear.
  
  This is what I think might have happened: there's multiple VCFs against the same reference I assume these were loaded and a database `join` operation performed using the reference genome column 'REF' as the index for the `join` across all input VCFs. I've placed this here because I'm not 100% sure what the input file represents - if it's not what I think it is - thats how I'd get to the allelic frequency.
  
  Why do I think that?
  
  1. Without multiple strains the allelic frequency cannot be calculated.
  2. The file states '200 samples'
  3. The presence of multiple alleles in the 'ALT' column.
  
  The problems are:
  
  * heterozygosity. This assumes falciparum is in its haploid state (it has a diploid stage). If its the diploid stage the allelic frequency cannot be calculated using this input file because the linkage was discarded (which is really important)
  * The underlying problem is the number of actual strains isn't explicitly stated .
  * the frequency of the 'ALT' doesn't appear to be declared
  * even if the a frequency was declared (but not presented) that must be linked to the 'ALT' column, each allele needs a exact frequency, otherwise population genetics will not work on this data.
  
  **First conclusion** The allelic frequency is performed across each row using the reference genome as the reference via `value_counts()` divided by the number of strains and presented in its own column. Somehow the relevant columns are missing - will need to see them.
  
  **However** Again on the information presented I don't think it can be done because if this represents 200 samples the frequency of a given allele is not declared in this data, or simply the number of 'ALT's across all VCFs. Without accessing that number the allelic frequency is not doable because it's entirely dependent on it.
  
  **Possible alternative conclusion** Its possible the question needs to be redefined not as *allelic frequency* - this has an exact definition within population genetics, but *genetic diversity*, i.e. remove the term 'frequency' altogether. What I think is being calculated is genetic diversity of all samples in the file against a reference genome. If that is correct it would be good to change the question accordingly. These are different mathematical entities.
  
  Whatever operation is required is easy to calculate, but its obtaining that information from the input file and the precise nature of the investigation.
  
  
  ---
  
  