Answers.Body: <p>Here is a quick and kind of dirty solution with minimum intervention to the code:</p>
  
  <ul>
  <li>Add <code>$outname'\t'</code> right after <code>echo -e</code> to two two lines, the lines after <code>#Header for output file</code> and <code>#Print output</code>. The former will alter your header and the latter your actual output: you will have a first column with file name information.</li>
  <li>Then you can run this script in a for loop to generate multiple output files which then can be merged into a single file with <code>cat</code>.</li>
  </ul>
  
  <p>Note that with this workaround, you will have multiple headers in the merged file. If you need a clean output, you will need to put your while loop in a for loop.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T14:43:50.067Z
Answers.Text: Thank you so much @haci . I edited the script as you suggested adding for loop to that in my main post but I obtained the same error posted in my main post
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T15:10:00.893Z
Answers.Text: What happens when you get rid of the whitespace in the `$outname'\t' $chrom` bit (remember there are two of these for the two lines that were altered from the original code) and the quotes as @terdon suggested in their comment?
Answers.UserDisplayName: haci
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T15:40:52.077Z
Answers.Text: Sorry I tried all of your comments but I failed I have attached my original script and two vcf files here maybe it is better than pasting the code here
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T11:06:24.120Z
Answers.LastActivityDate: 2020-01-01T11:06:24.120Z
Answers.UserDisplayName: haci
Answers.Body: <p>Using shell loops for text practice <a href="https://unix.stackexchange.com/q/169716/22222">is considered bad practice</a>. It is exceedingly slow, the syntax is complicated so it's very easy to get it wrong and it's just painful. The shell isn't designed as a proper scripting language, so while it can be (ab)used that way, you really should avoid it whenever possible. </p>
  
  <p>Here's a Perl script that should do exactly what your bash script does but will be several times faster and can deal with multiple input files:</p>
  
  <pre class="lang-perl prettyprint-override"><code>#!/usr/bin/env perl
  
  ## Iterate over all files given at the command line
  foreach my <span class="math-container">$inFile (@ARGV) {
    print STDERR "Processing $inFile...\n";
    my $outFile = "$inFile.parsed.txt";
    $outFile =~ m|([^/]+)$|;
    #/#Useless comment to fix SE syntax highlighting
    my $outFileName = $1;
    open(my $outFileHandle, '&gt;', $outFile) or
        die("Couldn't open '$outFile' for writing: $!");
    open(my $inFileHandle, '&lt;', $inFile) or
        die("Couldn't open '$inFile' for writing: $</span>!");
  
    ## Print header
    print <span class="math-container">$outFileHandle "$</span>outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t" .
        "TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount" .
            "\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n";
    ## Read all lines of the input file
    while (&lt;<span class="math-container">$inFileHandle&gt;) {
      ## Skip headers
      next if /^#/;
      ## Get VCF fields
      my ($chrom, $pos, $name, $ref, $alt, $qual,
          $filter, $info, $format, $values) = split(/\t/);
      my @infoFields = split(/;/, $info);
      my $readCount = $infoFields[12];
      $</span>readCount =~ s/ReadCount=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my <span class="math-container">$variantAlleleCount = $</span>infoFields[25];
      <span class="math-container">$variantAlleleCount =~ s/VariantAlleleCount=//;
      #/#Useless comment to fix SE syntax highlighting
      my $</span>referenceAlleleCount= <span class="math-container">$readCount - $</span>variantAlleleCount;
  
      my <span class="math-container">$readCountControl = $</span>infoFields[13];
      $readCountControl =~ s/ReadCountControl=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my <span class="math-container">$variantAlleleCountControl = $</span>infoFields[26];
      $variantAlleleCountControl =~ s/VariantAlleleCountControl=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my <span class="math-container">$referenceAlleleCountControl = $</span>readCountControl - <span class="math-container">$variantAlleleCountControl;
      my $</span>VAF = <span class="math-container">$infoFields[27];
      $</span>VAF =~ s/VariantAlleleFrequency=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my <span class="math-container">$outString = join("\t", $</span>outFileName, <span class="math-container">$chrom, $</span>pos, <span class="math-container">$ref, $</span>alt, <span class="math-container">$readCount,
                           $</span>variantAlleleCount, <span class="math-container">$referenceAlleleCount, $</span>readCountControl,
                           <span class="math-container">$variantAlleleCountControl, $</span>referenceAlleleCountControl, <span class="math-container">$VAF);
      print $</span>outFileHandle "<span class="math-container">$outString\n";
    }
    close($</span>outFileHandle);
    close($inFileHandle);
  }
  </code></pre>
  
  <p>Save that file as <code>parser.pl</code> and then run it on all your vcf files with:</p>
  
  <pre><code>perl parser.pl /path/to/vcfs/*vcf
  </code></pre>
  
  <p>That should create a <code>.parsed.txt</code> file for each input file in the target directory. </p>
  
  <hr>
  
  <p>If you just have to use the shell for this, you can try with this improved version of your script:</p>
  
  <pre class="lang-sh prettyprint-override"><code>#!/usr/bin/env bash
  
  for file in "<span class="math-container">$@"; do
    outFile="$</span>{file}.parsed.txt"
    outFileName=<span class="math-container">$(basename "$</span>outFile")
    (
      printf '%s\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t' "<span class="math-container">$outFileName"
      printf 'TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount\t'
      printf 'NormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n'
    ) &gt; "$</span>outFile"
  
    echo "Reading <span class="math-container">$file"
    grep -v '^#' "$</span>file" |
      while read -r line; do
        read chrom Pos Ref Alt rest &lt; &lt;(awk -F"\t" '{print <span class="math-container">$1,$</span>2,<span class="math-container">$4,$</span>5}' &lt;&lt;&lt;"<span class="math-container">$line")
        ReadCount=$</span>(echo "<span class="math-container">$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )
        VariantAlleleCount=$</span>(echo "<span class="math-container">$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')
        ReferenceAlleleCount=$</span>((<span class="math-container">$ReadCount - $</span>VariantAlleleCount))
  
        ReadCountControl=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')
        VariantAlleleCountControl=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   
  
        ReferenceAlleleCountControl=<span class="math-container">$(($</span>ReadCountControl - $VariantAlleleCountControl))
  
        VAF=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')
        printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "<span class="math-container">$outFileName" "$</span>chrom" "<span class="math-container">$Pos" "$</span>Ref" "<span class="math-container">$Alt" \
               "$</span>ReadCount" "<span class="math-container">$VariantAlleleCount" "$</span>ReferenceAlleleCount" "<span class="math-container">$ReadCountControl" "$</span>VariantAlleleCountControl" \
               "<span class="math-container">$ReferenceAlleleCountControl" "$</span>VAF" &gt;&gt; "$outFile" 
      done
  done
  </code></pre>
  
  <hr>
  
  <p>Just to illustrate what I mean about the shell being slow, I ran both scripts above on the example files you provided:</p>
  
  <pre><code>$ time foo.pl *vcf 2&gt;/dev/null
  
  real    0m0.051s
  user    0m0.047s
  sys     0m0.003s
  
  $ time bar.sh *vcf 2&gt;/dev/null
  
  real    2m28.842s
  user    3m33.648s
  sys     1m12.456s
  </code></pre>
  
  <p>As you can see above, the perl script took less than a second, while the bash script took almost two and a half minutes!</p>
  
  <hr>
  
  <p>Finally, if you find the perl script too long, here's the same thing in a more condensed style:</p>
  
  <pre class="lang-perl prettyprint-override"><code>#!/usr/bin/env perl
  
  ## Iterate over all files given at the command line
  foreach my <span class="math-container">$inFile (@ARGV) {
    print STDERR "Processing $inFile...\n";
    my $outFile = "$inFile.parsed.txt";
    $outFile =~ m|([^/]+)$|;
    #/#Useless comment to fix SE syntax highlighting
    my $outFileName = $1;
    open(my $outFileHandle, '&gt;', $outFile);
    open(my $inFileHandle, '&lt;', $</span>inFile);
  
    ## Print header
    print <span class="math-container">$outFileHandle "$</span>outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t" .
        "TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount" .
            "\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n";
    ## Read all lines of the input file
    while (&lt;<span class="math-container">$inFileHandle&gt;) {
      ## Skip headers
      next if /^#/;
      ## Get VCF fields
      my ($chrom, $pos, $name, $ref, $alt, $qual,
          $filter, $info, $format, $values) = split(/\t/);
      $info=~s/[^;]+=//g;
      my @infoFields = split(/;/, $info);
      print $outFileHandle (join "\t", $outFileName, $chrom, $pos, $ref, $alt,
                                 $infoFields[12],$infoFields[25],
                                 $infoFields[12]-$infoFields[25],
                                 $infoFields[13],$infoFields[26],
                                 $infoFields[13]-$infoFields[26],
                                 $</span>infoFields[27]) . "\n";
    }
  }
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T17:16:51.280Z
Answers.Text: Oh my God, this is amazing. I am wondering when some people put their time and efforts to helping the other people   this nicely . The perl done that in a second
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-03T04:56:49.297Z
Answers.Text: Sorry @terdon your code works on Single Nucleotide variants (SNVs) calls very well but on INDELs returns empty fields that may be because for example the position of ReferenceAlleleCount in SNV and INDEL are not the same. This is link to one of my INDEL files coming from the same Strelka variant caller https://sotonac-my.sharepoint.com/personal/fi1d18_soton_ac_uk/Documents/Microsoft%20Teams%20Chat%20Files/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.passed.somatic.indels.vcf I am not sure even I can post this comment or not because surely I see myself too annoying
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-03T09:15:19.170Z
Answers.Text: @Angel I can't access that file. It's probably best if you ask a new question, and post some lines from the bad files. Or, if the position is different, just try modifying the script and changing the numbers in the various `$infofields[N]`.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-04T16:40:49.590Z
Answers.Text: Thank you @terdon, I have asked this as a new question here https://bioinformatics.stackexchange.com/questions/11074/looping-over-several-files-in-bash  as I found the script specific to INDELs differs to what works for SNVs. If possible have a look please; thank you in advance
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-04T16:43:55.430Z
Answers.Text: @Angel you will need to explain what information you need. Since your script doesn't work, that won't help us understand your objective. So, show us your expected output from a few lines of example input.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-04T18:30:28.283Z
Answers.Text: Thank you so much for considering my issue. Actually all I want is Variant Allele Frequency (VAF) which is obtained by Depth of Alternative Allele / Total Depth or AD/DP. I have addressed this in my asked main post.
Answers.UserDisplayName: Zizogolu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-01T17:03:10.967Z
Answers.LastActivityDate: 2020-01-01T17:03:10.967Z
Answers.UserDisplayName: terdon
Body: <p>I have a bash script which extracts some information from a .vcf file but one .vcf file at each time. How I can change this script in a way to work on a bunch of .vcf files and the output is a .txt file named by the name of original .vcf</p>
  
  <p>This is my script </p>
  
  <p>help please</p>
  
  <hr>
  
  <blockquote>
    <p>EDITED</p>
  </blockquote>
  
  <pre><code> #!/usr/bash
  
  #outfilename
  outname=$".parsed.txt"
  
  #Header for output file
  echo -e "<span class="math-container">$outname\t$</span>chrom\t<span class="math-container">$Pos\t$</span>Ref\t<span class="math-container">$Alt\t$</span>ReadCount\t<span class="math-container">$VariantAlleleCount\t$</span>ReferenceAlleleCount\t<span class="math-container">$ReadCountControl\t$</span>VariantAlleleCountControl\t<span class="math-container">$ReferenceAlleleCountControl\t$</span>VAF" &gt;&gt; $outname ;
  while read -r line ; 
  do
  
  #Basic information
  chrom=<span class="math-container">$(echo "$</span>line" | sed 's/ /\t/g' | cut -f 1) #&amp;&amp; echo <span class="math-container">$chrom;
  Pos=$</span>(echo "<span class="math-container">$line" | sed 's/ /\t/g' |  cut -f 2) #&amp;&amp; echo $</span>Pos;
  Ref=<span class="math-container">$(echo "$</span>line" | sed 's/ /\t/g' | cut -f 4)
  Alt=<span class="math-container">$(echo "$</span>line" | sed 's/ /\t/g' | cut -f 5)
  
  #Tumor sample read, variant and reference information
  ReadCount=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )
  VariantAlleleCount=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')
  ReferenceAlleleCount=<span class="math-container">$(echo $</span> line | awk -v rc="<span class="math-container">$ReadCount" -v vac="$</span>VariantAlleleCount" '{print rc-vac}')
  
  
  #Control or Normal read, variant, reference information
  ReadCountControl=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')
  VariantAlleleCountControl=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   
  
  ReferenceAlleleCountControl=<span class="math-container">$(echo "$</span>line" | awk -v rcc="<span class="math-container">$ReadCountControl" -v vacc="$</span>VariantAlleleCountControl" '{print rcc-vacc}')
  
  
  VAF=<span class="math-container">$(echo "$</span>line" | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')
  #Print output
  echo -e "<span class="math-container">$outname\t""$</span>chrom"'\t'"<span class="math-container">$Pos"'\t'"$</span>Ref"'\t'"<span class="math-container">$Alt"'\t'"$</span>ReadCount"'\t'"<span class="math-container">$VariantAlleleCount"'\t'"$</span>ReferenceAlleleCount"'\t'"<span class="math-container">$ReadCountControl"'\t'"$</span>VariantAlleleCountControl"'\t'"<span class="math-container">$ReferenceAlleleCountControl"'\t'"$</span>VAF" &gt;&gt; $outname ;
  
  #Remove info tags from VCF
  done; &lt; &lt;( egrep -v '#' "$1")
  </code></pre>
  
  <hr>
  
  <p>And the error I obtained</p>
  
  <pre><code>[fi1d18@cyan01 new]<span class="math-container">$ bash vcf_parasing.sh
  vcf_parasing.sh: line 7: syntax error near unexpected token `echo'
  vcf_parasing.sh: line 7: `echo -e $</span>outname'\t' "Chrom"'\t'"Position"'\t'"Ref"'\t'"Alt"'\t'"TumorReadCount"'\t'"TumorVariantAlleleCount"'\t'"TumorReferenceAlleleCount"'\t'"NormalReadCount"'\t'"NormalVariantAlleleCount"'\t'"NormalReferenceAlleleCount"'\t'"VAF" &gt; <span class="math-container">$outname'
  [fi1d18@cyan01 new]$</span>
  </code></pre>
  
  <p>My script</p>
  
  <p><a href="https://www.dropbox.com/s/di49mij2a43i43s/vcf_parasing.sh?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/s/di49mij2a43i43s/vcf_parasing.sh?dl=0</a></p>
  
  <p>Two samples of .vcf</p>
  
  <p><a href="https://www.dropbox.com/s/ryk7q0aybeo5cy2/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.snp.pass.vcf?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/s/ryk7q0aybeo5cy2/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.snp.pass.vcf?dl=0</a></p>
  
  <p><a href="https://www.dropbox.com/s/u44s6d54r5d57bu/LP6008336-DNA_H02_vs_LP6008333-DNA_H02.snp.pass.vcf?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/s/u44s6d54r5d57bu/LP6008336-DNA_H02_vs_LP6008333-DNA_H02.snp.pass.vcf?dl=0</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-01T13:30:21.553Z
Comments.Text: Doing this sort of thing with a bash loop over every line is a really, really bad idea. It will be slower than just about any other method, the syntax is hard to read (and I say this as someone with considerable experience with shell scripts) and the whole thing will be more painful than needed. If you could [edit] your question and explain exactly what information you are trying to extract, I'm sure we could help you find a better way.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-01T14:46:41.997Z
Comments.Text: Thank you @terdon. This script extract some fields from .vcf file output of Strelka mutation caller. the problem is that except for the some columns, the location of the other information is of easily detecting
Comments.UserDisplayName: Zizogolu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-01T14:59:14.037Z
Comments.Text: Note that you don't need all of those quotes for the echo. Just use `echo -e "$outname\t$chrom\t$Pos\t$Ref\t$Alt\t$ReadCount\t$VariantAlleleCount\t$ReferenceAlleleCount\t$ReadCountControl\t$VariantAlleleCountControl\t$ReferenceAlleleCountControl\t$VAF" >> $outname ;`
Comments.UserDisplayName: terdon
Tags: <vcf><bash><linux>
Title: Changing this code in a way to work for my files
ThreadText: # Changing this code in a way to work for my files
  
  (Asked by: username_0 on 2019-12-31)
  
  I have a bash script which extracts some information from a .vcf file but one .vcf file at each time. How I can change this script in a way to work on a bunch of .vcf files and the output is a .txt file named by the name of original .vcf
  
  This is my script
  
  help please
  
  ---
  
  > EDITED
  
  ```
   #!/usr/bash
  
  #outfilename
  outname=$".parsed.txt"
  
  #Header for output file
  echo -e "$outname\t$chrom\t$Pos\t$Ref\t$Alt\t$ReadCount\t$VariantAlleleCount\t$ReferenceAlleleCount\t$ReadCountControl\t$VariantAlleleCountControl\t$ReferenceAlleleCountControl\t$VAF" >> $outname ;
  while read -r line ; 
  do
  
  #Basic information
  chrom=$(echo "$line" | sed 's/ /\t/g' | cut -f 1) #&& echo $chrom;
  Pos=$(echo "$line" | sed 's/ /\t/g' |  cut -f 2) #&& echo $Pos;
  Ref=$(echo "$line" | sed 's/ /\t/g' | cut -f 4)
  Alt=$(echo "$line" | sed 's/ /\t/g' | cut -f 5)
  
  #Tumor sample read, variant and reference information
  ReadCount=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )
  VariantAlleleCount=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')
  ReferenceAlleleCount=$(echo $ line | awk -v rc="$ReadCount" -v vac="$VariantAlleleCount" '{print rc-vac}')
  
  #Control or Normal read, variant, reference information
  ReadCountControl=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')
  VariantAlleleCountControl=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   
  
  ReferenceAlleleCountControl=$(echo "$line" | awk -v rcc="$ReadCountControl" -v vacc="$VariantAlleleCountControl" '{print rcc-vacc}')
  
  VAF=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')
  #Print output
  echo -e "$outname\t""$chrom"'\t'"$Pos"'\t'"$Ref"'\t'"$Alt"'\t'"$ReadCount"'\t'"$VariantAlleleCount"'\t'"$ReferenceAlleleCount"'\t'"$ReadCountControl"'\t'"$VariantAlleleCountControl"'\t'"$ReferenceAlleleCountControl"'\t'"$VAF" >> $outname ;
  
  #Remove info tags from VCF
  done; < <( egrep -v '#' "$1")
  
  ```
  
  ---
  
  And the error I obtained
  
  ```
  [fi1d18@cyan01 new]$ bash vcf_parasing.sh
  vcf_parasing.sh: line 7: syntax error near unexpected token `echo'
  vcf_parasing.sh: line 7: `echo -e $outname'\t' "Chrom"'\t'"Position"'\t'"Ref"'\t'"Alt"'\t'"TumorReadCount"'\t'"TumorVariantAlleleCount"'\t'"TumorReferenceAlleleCount"'\t'"NormalReadCount"'\t'"NormalVariantAlleleCount"'\t'"NormalReferenceAlleleCount"'\t'"VAF" > $outname'
  [fi1d18@cyan01 new]$
  
  ```
  
  My script
  
  <https://www.dropbox.com/s/di49mij2a43i43s/vcf_parasing.sh?dl=0>
  
  Two samples of .vcf
  
  <https://www.dropbox.com/s/ryk7q0aybeo5cy2/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.snp.pass.vcf?dl=0>
  
  <https://www.dropbox.com/s/u44s6d54r5d57bu/LP6008336-DNA_H02_vs_LP6008333-DNA_H02.snp.pass.vcf?dl=0>
  
  **Tags:** vcf, bash, linux
  
  > **Comments:**
  > **username_1**: Doing this sort of thing with a bash loop over every line is a really, really bad idea. It will be slower than just about any other method, the syntax is hard to read (and I say this as someone with considerable experience with shell scripts) and the whole thing will be more painful than needed. If you could [edit] your question and explain exactly what information you are trying to extract, I'm sure we could help you find a better way.  
  > **username_0**: Thank you @username_1. This script extract some fields from .vcf file output of Strelka mutation caller. the problem is that except for the some columns, the location of the other information is of easily detecting  
  > **username_1**: Note that you don't need all of those quotes for the echo. Just use `echo -e "$outname\t$chrom\t$Pos\t$Ref\t$Alt\t$ReadCount\t$VariantAlleleCount\t$ReferenceAlleleCount\t$ReadCountControl\t$VariantAlleleCountControl\t$ReferenceAlleleCountControl\t$VAF" >> $outname ;`  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-01-01. Score: 5 | Accepted answer
  
  Using shell loops for text practice [is considered bad practice](https://unix.stackexchange.com/q/169716/22222). It is exceedingly slow, the syntax is complicated so it's very easy to get it wrong and it's just painful. The shell isn't designed as a proper scripting language, so while it can be (ab)used that way, you really should avoid it whenever possible.
  
  Here's a Perl script that should do exactly what your bash script does but will be several times faster and can deal with multiple input files:
  
  ```perl
  #!/usr/bin/env perl
  
  ## Iterate over all files given at the command line
  foreach my $inFile (@ARGV) {
    print STDERR "Processing $inFile...\n";
    my $outFile = "$inFile.parsed.txt";
    $outFile =~ m|([^/]+)$|;
    #/#Useless comment to fix SE syntax highlighting
    my $outFileName = $1;
    open(my $outFileHandle, '>', $outFile) or
        die("Couldn't open '$outFile' for writing: $!");
    open(my $inFileHandle, '<', $inFile) or
        die("Couldn't open '$inFile' for writing: $!");
  
    ## Print header
    print $outFileHandle "$outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t" .
        "TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount" .
            "\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n";
    ## Read all lines of the input file
    while (<$inFileHandle>) {
      ## Skip headers
      next if /^#/;
      ## Get VCF fields
      my ($chrom, $pos, $name, $ref, $alt, $qual,
          $filter, $info, $format, $values) = split(/\t/);
      my @infoFields = split(/;/, $info);
      my $readCount = $infoFields[12];
      $readCount =~ s/ReadCount=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my $variantAlleleCount = $infoFields[25];
      $variantAlleleCount =~ s/VariantAlleleCount=//;
      #/#Useless comment to fix SE syntax highlighting
      my $referenceAlleleCount= $readCount - $variantAlleleCount;
  
      my $readCountControl = $infoFields[13];
      $readCountControl =~ s/ReadCountControl=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my $variantAlleleCountControl = $infoFields[26];
      $variantAlleleCountControl =~ s/VariantAlleleCountControl=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my $referenceAlleleCountControl = $readCountControl - $variantAlleleCountControl;
      my $VAF = $infoFields[27];
      $VAF =~ s/VariantAlleleFrequency=//;
      #/#Useless comment to fix SE syntax highlighting
  
      my $outString = join("\t", $outFileName, $chrom, $pos, $ref, $alt, $readCount,
                           $variantAlleleCount, $referenceAlleleCount, $readCountControl,
                           $variantAlleleCountControl, $referenceAlleleCountControl, $VAF);
      print $outFileHandle "$outString\n";
    }
    close($outFileHandle);
    close($inFileHandle);
  }
  
  ```
  
  Save that file as `parser.pl` and then run it on all your vcf files with:
  
  ```
  perl parser.pl /path/to/vcfs/*vcf
  
  ```
  
  That should create a `.parsed.txt` file for each input file in the target directory.
  
  ---
  
  If you just have to use the shell for this, you can try with this improved version of your script:
  
  ```sh
  #!/usr/bin/env bash
  
  for file in "$@"; do
    outFile="${file}.parsed.txt"
    outFileName=$(basename "$outFile")
    (
      printf '%s\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t' "$outFileName"
      printf 'TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount\t'
      printf 'NormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n'
    ) > "$outFile"
  
    echo "Reading $file"
    grep -v '^#' "$file" |
      while read -r line; do
        read chrom Pos Ref Alt rest < <(awk -F"\t" '{print $1,$2,$4,$5}' <<<"$line")
        ReadCount=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )
        VariantAlleleCount=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')
        ReferenceAlleleCount=$(($ReadCount - $VariantAlleleCount))
  
        ReadCountControl=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')
        VariantAlleleCountControl=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   
  
        ReferenceAlleleCountControl=$(($ReadCountControl - $VariantAlleleCountControl))
  
        VAF=$(echo "$line" | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')
        printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$outFileName" "$chrom" "$Pos" "$Ref" "$Alt" \
               "$ReadCount" "$VariantAlleleCount" "$ReferenceAlleleCount" "$ReadCountControl" "$VariantAlleleCountControl" \
               "$ReferenceAlleleCountControl" "$VAF" >> "$outFile" 
      done
  done
  
  ```
  
  ---
  
  Just to illustrate what I mean about the shell being slow, I ran both scripts above on the example files you provided:
  
  ```
  $ time foo.pl *vcf 2>/dev/null
  
  real    0m0.051s
  user    0m0.047s
  sys     0m0.003s
  
  $ time bar.sh *vcf 2>/dev/null
  
  real    2m28.842s
  user    3m33.648s
  sys     1m12.456s
  
  ```
  
  As you can see above, the perl script took less than a second, while the bash script took almost two and a half minutes!
  
  ---
  
  Finally, if you find the perl script too long, here's the same thing in a more condensed style:
  
  ```perl
  #!/usr/bin/env perl
  
  ## Iterate over all files given at the command line
  foreach my $inFile (@ARGV) {
    print STDERR "Processing $inFile...\n";
    my $outFile = "$inFile.parsed.txt";
    $outFile =~ m|([^/]+)$|;
    #/#Useless comment to fix SE syntax highlighting
    my $outFileName = $1;
    open(my $outFileHandle, '>', $outFile);
    open(my $inFileHandle, '<', $inFile);
  
    ## Print header
    print $outFileHandle "$outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t" .
        "TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount" .
            "\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n";
    ## Read all lines of the input file
    while (<$inFileHandle>) {
      ## Skip headers
      next if /^#/;
      ## Get VCF fields
      my ($chrom, $pos, $name, $ref, $alt, $qual,
          $filter, $info, $format, $values) = split(/\t/);
      $info=~s/[^;]+=//g;
      my @infoFields = split(/;/, $info);
      print $outFileHandle (join "\t", $outFileName, $chrom, $pos, $ref, $alt,
                                 $infoFields[12],$infoFields[25],
                                 $infoFields[12]-$infoFields[25],
                                 $infoFields[13],$infoFields[26],
                                 $infoFields[13]-$infoFields[26],
                                 $infoFields[27]) . "\n";
    }
  }
  
  ```
  
  > **Comments:**
  > **username_0**: Oh my God, this is amazing. I am wondering when some people put their time and efforts to helping the other people   this nicely . The perl done that in a second  
  > **username_0**: Sorry @username_1 your code works on Single Nucleotide variants (SNVs) calls very well but on INDELs returns empty fields that may be because for example the position of ReferenceAlleleCount in SNV and INDEL are not the same. This is link to one of my INDEL files coming from the same Strelka variant caller https://sotonac-my.sharepoint.com/personal/fi1d18_soton_ac_uk/Documents/Microsoft%20Teams%20Chat%20Files/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.passed.somatic.indels.vcf I am not sure even I can post this comment or not because surely I see myself too annoying  
  > **username_1**: @Angel I can't access that file. It's probably best if you ask a new question, and post some lines from the bad files. Or, if the position is different, just try modifying the script and changing the numbers in the various `$infofields[N]`.  
  > **username_0**: Thank you @username_1, I have asked this as a new question here https://bioinformatics.stackexchange.com/questions/11074/looping-over-several-files-in-bash  as I found the script specific to INDELs differs to what works for SNVs. If possible have a look please; thank you in advance  
  > **username_1**: @Angel you will need to explain what information you need. Since your script doesn't work, that won't help us understand your objective. So, show us your expected output from a few lines of example input.  
  > **username_0**: Thank you so much for considering my issue. Actually all I want is Variant Allele Frequency (VAF) which is obtained by Depth of Alternative Allele / Total Depth or AD/DP. I have addressed this in my asked main post.  
  
  ---
  
  ### Answer by username_2 on 2020-01-01. Score: 1
  
  Here is a quick and kind of dirty solution with minimum intervention to the code:
  
  * Add `$outname'\t'` right after `echo -e` to two two lines, the lines after `#Header for output file` and `#Print output`. The former will alter your header and the latter your actual output: you will have a first column with file name information.
  * Then you can run this script in a for loop to generate multiple output files which then can be merged into a single file with `cat`.
  
  Note that with this workaround, you will have multiple headers in the merged file. If you need a clean output, you will need to put your while loop in a for loop.
  
  > **Comments:**
  > **username_0**: Thank you so much @username_2 . I edited the script as you suggested adding for loop to that in my main post but I obtained the same error posted in my main post  
  > **username_2**: What happens when you get rid of the whitespace in the `$outname'\t' $chrom` bit (remember there are two of these for the two lines that were altered from the original code) and the quotes as @username_1 suggested in their comment?  
  > **username_0**: Sorry I tried all of your comments but I failed I have attached my original script and two vcf files here maybe it is better than pasting the code here  
  
  ---
  
  