Answers.Body: <p>Another awk approach. This should work on a normal VCF file (it will print the headers) as well as your example and only changes the ALT field if its first character is the same as the 2nd field:</p>
  <pre><code><span class="math-container">$ awk -F"\t" -vOFS="\t" '{ if(substr($5,1,1) == $4){$5=substr($</span>5,2); }}1' ff.vcf 
  #CHROM  POS ID  REF ALT
  chr4    5708160 .   G   ATAAC
  chr4    30280292    .   C   ACGTAT
  chr6    166253422   .   C   CTTTCT
  chr11   21931340    .   A   ACACACGC
  </code></pre>
  <p>To run this on all of your files, assuming they are the only files in the current directory with a <code>.txt</code> extension, you can do:</p>
  <pre><code>for file in *txt; do
      awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; '{ if(substr(<span class="math-container">$5,1,1) == $</span>4){<span class="math-container">$5=substr($</span>5,2); }}1' &quot;<span class="math-container">$file" &gt; "fixed.$</span>file&quot;
  done
  </code></pre>
  <p>This will loop over all <code>*.txt</code> files in the current directory. For each <code>foo.txt</code>, it will create <code>fixed.foo.txt</code> with the modified data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-04T16:56:01.950Z
Answers.LastActivityDate: 2021-05-23T13:40:06.183Z
Answers.LastEditDate: 2021-05-23T13:40:06.183Z
Answers.UserDisplayName: terdon
Answers.Body: <p>I am making a couple of assumptions here: </p>
  
  <ol>
  <li>the delimiter is tab</li>
  <li>the reference allele is always present in the ALT column</li>
  <li>the reference allele in the ALT column is always in the first position</li>
  </ol>
  
  <p>With those in mind, you can do this using <code>awk</code> as follows: </p>
  
  <pre><code>awk 'BEGIN{FS="\t";OFS="\t"}{$3=substr($3,2)}{print $0}'
  </code></pre>
  
  <p>Update 1: Using the new VCF style file that @Angel provided, here's how I would do it. This is not significantly different from what @terdon proposed below. The code below will work even if the reference allele is >1 nt.</p>
  
  <pre><code>$ cat test.vcf
  #CHROM  POS     ID      REF     ALT
  chr4    5708160 .       G       GATAAC
  chr4    30280292        .       C       CACGTAT
  chr6    166253422       .       C       CCTTTCT
  chr11   21931340        .       A       AACACACGC
  
  <span class="math-container">$ awk 'BEGIN{FS="\t";OFS="\t"}{l=length($4); if ($4==substr($5,1,l)) {$5=substr($5,l+1); print $0} else {print $</span>0}}' test.vcf
  #CHROM  POS     ID      REF     ALT
  chr4    5708160 .       G       ATAAC
  chr4    30280292        .       C       ACGTAT
  chr6    166253422       .       C       CTTTCT
  chr11   21931340        .       A       ACACACGC
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-04T17:55:07.527Z
Answers.Text: Ooh, of course it's different to mine! You are being clever and looking for the entire ref allele instead of assuming it is one residue. Nice! You might want to add an example with a multi-nt ref allele so people can see that your solution deals with those.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-04T16:51:04.373Z
Answers.LastActivityDate: 2019-12-04T17:41:49.570Z
Answers.LastEditDate: 2019-12-04T17:41:49.570Z
Answers.UserDisplayName: vkkodali
Body: <p>I have 200 <code>.txt</code> files on a Linux system. In each file, I have the reference allele in the <code>REF</code> column and INDELs in the <code>ALT</code> column.
  Let's say I have <code>A</code> in the REF column and <code>AC</code> in the ALT. I want to remove  the <code>A</code> from the ALT column because that is a repetition of the <code>A</code> reference allele. For example, if I have <code>C</code> in the REF column and <code>CCAT</code> in the ALT column, I only want CAT to remain in the ALT column.</p>
  
  <p>This is what my files look like</p>
  
  <pre><code>CHROM   REF ALT
  chr1    C   CT
  chr1    C   CT
  chr1    C   CA
  chr1    C   CT
  </code></pre>
  
  <p>Can someone help me please?</p>
  
  <p>Solution from @Terdon</p>
  
  <p>This is my original file</p>
  
  <pre><code>#CHROM  POS ID  REF ALT
  chr4    5708160 .   G   GATAAC
  chr4    30280292    .   C   CACGTAT
  chr6    166253422   .   C   CCTTTCT
  chr11   21931340    .   A   AACACACGC
  </code></pre>
  
  <p>This is what happened after running your kindly suggestion </p>
  
  <pre><code>[fi1d18@cyan01 indel]<span class="math-container">$ awk -F"\t" -vOFS="\t" '{ if(substr($5,1,1) == $4){$5=substr($</span>5,4); }}1' indels.vcf &gt; out.vcf
  
  #CHROM  POS ID  REF ALT
  chr4    5708160 .   G   AAC
  chr4    30280292    .   C   GTAT
  chr6    166253422   .   C   TTCT
  </code></pre>
  
  <p>I think more than one allele being removed from ALT column, is not it?</p>
  
Tags: <vcf><text-processing>
Title: Remove repeated ALT allele from the REF field of a vcf-like file
ThreadText: # Remove repeated ALT allele from the REF field of a vcf-like file
  
  (Asked by: username_0 on 2019-12-04)
  
  I have 200 `.txt` files on a Linux system. In each file, I have the reference allele in the `REF` column and INDELs in the `ALT` column.
  Let's say I have `A` in the REF column and `AC` in the ALT. I want to remove the `A` from the ALT column because that is a repetition of the `A` reference allele. For example, if I have `C` in the REF column and `CCAT` in the ALT column, I only want CAT to remain in the ALT column.
  
  This is what my files look like
  
  ```
  CHROM   REF ALT
  chr1    C   CT
  chr1    C   CT
  chr1    C   CA
  chr1    C   CT
  
  ```
  
  Can someone help me please?
  
  Solution from @username_1
  
  This is my original file
  
  ```
  #CHROM  POS ID  REF ALT
  chr4    5708160 .   G   GATAAC
  chr4    30280292    .   C   CACGTAT
  chr6    166253422   .   C   CCTTTCT
  chr11   21931340    .   A   AACACACGC
  
  ```
  
  This is what happened after running your kindly suggestion
  
  ```
  [fi1d18@cyan01 indel]$ awk -F"\t" -vOFS="\t" '{ if(substr($5,1,1) == $4){$5=substr($5,4); }}1' indels.vcf > out.vcf
  
  #CHROM  POS ID  REF ALT
  chr4    5708160 .   G   AAC
  chr4    30280292    .   C   GTAT
  chr6    166253422   .   C   TTCT
  
  ```
  
  I think more than one allele being removed from ALT column, is not it?
  
  **Tags:** vcf, text-processing
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-12-04. Score: 4 | Accepted answer
  
  I am making a couple of assumptions here:
  
  1. the delimiter is tab
  2. the reference allele is always present in the ALT column
  3. the reference allele in the ALT column is always in the first position
  
  With those in mind, you can do this using `awk` as follows:
  
  ```
  awk 'BEGIN{FS="\t";OFS="\t"}{$3=substr($3,2)}{print $0}'
  
  ```
  
  Update 1: Using the new VCF style file that @Angel provided, here's how I would do it. This is not significantly different from what @username_1 proposed below. The code below will work even if the reference allele is >1 nt.
  
  ```
  $ cat test.vcf
  #CHROM  POS     ID      REF     ALT
  chr4    5708160 .       G       GATAAC
  chr4    30280292        .       C       CACGTAT
  chr6    166253422       .       C       CCTTTCT
  chr11   21931340        .       A       AACACACGC
  
  $ awk 'BEGIN{FS="\t";OFS="\t"}{l=length($4); if ($4==substr($5,1,l)) {$5=substr($5,l+1); print $0} else {print $0}}' test.vcf
  #CHROM  POS     ID      REF     ALT
  chr4    5708160 .       G       ATAAC
  chr4    30280292        .       C       ACGTAT
  chr6    166253422       .       C       CTTTCT
  chr11   21931340        .       A       ACACACGC
  
  ```
  
  > **Comments:**
  > **username_1**: Ooh, of course it's different to mine! You are being clever and looking for the entire ref allele instead of assuming it is one residue. Nice! You might want to add an example with a multi-nt ref allele so people can see that your solution deals with those.  
  
  ---
  
  ### Answer by username_1 on 2019-12-04. Score: 2
  
  Another awk approach. This should work on a normal VCF file (it will print the headers) as well as your example and only changes the ALT field if its first character is the same as the 2nd field:
  
  ```
  $ awk -F"\t" -vOFS="\t" '{ if(substr($5,1,1) == $4){$5=substr($5,2); }}1' ff.vcf 
  #CHROM  POS ID  REF ALT
  chr4    5708160 .   G   ATAAC
  chr4    30280292    .   C   ACGTAT
  chr6    166253422   .   C   CTTTCT
  chr11   21931340    .   A   ACACACGC
  
  ```
  
  To run this on all of your files, assuming they are the only files in the current directory with a `.txt` extension, you can do:
  
  ```
  for file in *txt; do
      awk -F"\t" -vOFS="\t" '{ if(substr($5,1,1) == $4){$5=substr($5,2); }}1' "$file" > "fixed.$file"
  done
  
  ```
  
  This will loop over all `*.txt` files in the current directory. For each `foo.txt`, it will create `fixed.foo.txt` with the modified data.
  
  
  ---
  
  