Answers.Body: <p>You have an extra <code>done</code>, instead you want:</p>
  
  <pre><code>for i in <span class="math-container">$(ls *.fq*.gz | cut -f1-5 -d"_" | uniq); do
      echo $</span>i
      bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38 -1 <span class="math-container">${i}_R1.fq.gz | samtools view -bS - &gt;$</span>{i}.bam
  done
  </code></pre>
  
  <p>As an aside, I'm not a big fan of piping <code>ls</code> to cut and then uniq. Why not instead:</p>
  
  <pre><code>for f in *.fq.gz; do
      sampleName=<span class="math-container">${f%%_R1.fq.gz}
      echo $</span>sampleName
      bowtie2 -q ...
  done
  </code></pre>
  
  <p>That will do the same thing, but (A) it's clearer what's going on and (B) it can be reused if your sample names change. Also, stop writing bash scripts and use snakemake or something like that instead, it scales much better and is less error-prone.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T08:25:39.890Z
Answers.Text: now after looking at my script number of times i figured out the "done" is extra .let me try your code. My shell scripting is very rudimentary so i use which i know is safe to save my self from screwing up .
Answers.UserDisplayName: kcm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T08:50:06.800Z
Answers.Text: i ran your first code the but i still get this error "Error: Must specify at least one read input with -U/-1/-2 (ERR): bowtie2-align exited with value 1" its  not able to recognize the input file, not sure what's wrong .
Answers.UserDisplayName: kcm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T09:17:19.140Z
Answers.Text: Try `-U` instead of `-1` in the bowtie2 command.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T09:43:14.033Z
Answers.Text: yes that was the mistake i was doing thank you very much
Answers.UserDisplayName: kcm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T16:53:07.010Z
Answers.Text: @krushnachChandra please read [this link](https://mywiki.wooledge.org/ParsingLs) to understand why you should basically _never_ try to parse the output of `ls`.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-09T16:08:13.337Z
Answers.Text: @terdon I'd not say "never parse ls", I'd say "Don't parse ls unless you know exactly what you're doing, especially the filenames involved". I do a lot of `for f in $(/bin/ls -1 <glob> | downstream_operations)` because I'm 100% certain that each value of `$f` plays well with the other parts in the code.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-09T16:28:48.287Z
Answers.Text: @RamRS well, parsing ls is asking for trouble and just unnecessary. `for f in <glob>` does exactly the same thing but without calling an external process and without running any risk of failure because of file names. So why use a riskier and less efficient approach when it doesn't provide any benefit at all?
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-09T22:29:09.977Z
Answers.Text: It's really useful when the `$()` feeding the loop involves a `uniq`, for example. It can't always be avoided. Whenever possible, use a glob. But don't have a hard constraint against `$(/bin/ls | downstream_stuff)`
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-10T05:07:20.073Z
Answers.Text: thats a life saver link
Answers.UserDisplayName: kcm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-08T08:13:07.967Z
Answers.LastActivityDate: 2019-06-08T08:13:07.967Z
Answers.UserDisplayName: Devon Ryan
Body: <p>These are my files </p>
  
  <pre><code>SO_8543_IP_3_ablt_R1.fq.gz
  SO_8543_IP_4_ablt_R1.fq.gz
  SO_8543_INPUT_1_ablt_R1.fq.gz  SO_8543_IP_5_ablt_R1.fq.gz
  SO_8543_INPUT_2_ablt_R1.fq.gz  SO_8543_IP_6_ablt_R1.fq.gz
  SO_8543_INPUT_3_ablt_R1.fq.gz  SO_8543_IP_7_ablt_R1.fq.gz
  SO_8543_INPUT_4_ablt_R1.fq.gz  SO_8543_IP_8_ablt_R1.fq.gz
  SO_8543_IP_1_ablt_R1.fq.gz     
  SO_8543_IP_2_ablt_R1.fq.gz
  </code></pre>
  
  <p>Im running the below line </p>
  
  <pre><code>for i in <span class="math-container">$(ls *.fq*.gz | cut -f1-5 -d"_" | uniq);do echo $</span>i; done
  
  bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38  -1 <span class="math-container">${i}_R1.fq.gz | samtools view -bS - &gt;$</span>{i%}.bam;
  
  done
  </code></pre>
  
  <p>So if i use run this </p>
  
  <pre><code>for i in <span class="math-container">$(ls *.fq*.gz | cut -f1-5 -d"_" | uniq);do echo $</span>i; done
  </code></pre>
  
  <p>I get files as such </p>
  
  <pre><code>SO_8543_INPUT_1_ablt
  SO_8543_INPUT_2_ablt
  SO_8543_INPUT_3_ablt
  SO_8543_INPUT_4_ablt
  SO_8543_IP_1_ablt
  SO_8543_IP_2_ablt
  SO_8543_IP_3_ablt
  SO_8543_IP_4_ablt
  SO_8543_IP_5_ablt
  SO_8543_IP_6_ablt
  SO_8543_IP_7_ablt
  SO_8543_IP_8_ablt
  </code></pre>
  
  <p>The part of the file name that is consistent is <code>_R1.fq.gz</code> which goes after the input argument. As these are single end files so only 1 im giving one end as input.</p>
  
  <p>Error message </p>
  
  <pre><code>***
  Error: Must specify at least one read input with -U/-1/-2
  (ERR): bowtie2-align exited with value 1
  ./Chip_SEQ.sh: line 25: syntax error near unexpected token `done'
  ./Chip_SEQ.sh: line 25: `done'
  </code></pre>
  
  <p>it looks like its not able to recognize as input .Any help or suggestion would be highly appreciated </p>
  
Tags: <shell>
Title: Regular expression struggle
ThreadText: # Regular expression struggle
  
  (Asked by: username_0 on 2019-06-08)
  
  These are my files
  
  ```
  SO_8543_IP_3_ablt_R1.fq.gz
  SO_8543_IP_4_ablt_R1.fq.gz
  SO_8543_INPUT_1_ablt_R1.fq.gz  SO_8543_IP_5_ablt_R1.fq.gz
  SO_8543_INPUT_2_ablt_R1.fq.gz  SO_8543_IP_6_ablt_R1.fq.gz
  SO_8543_INPUT_3_ablt_R1.fq.gz  SO_8543_IP_7_ablt_R1.fq.gz
  SO_8543_INPUT_4_ablt_R1.fq.gz  SO_8543_IP_8_ablt_R1.fq.gz
  SO_8543_IP_1_ablt_R1.fq.gz     
  SO_8543_IP_2_ablt_R1.fq.gz
  
  ```
  
  Im running the below line
  
  ```
  for i in $(ls *.fq*.gz | cut -f1-5 -d"_" | uniq);do echo $i; done
  
  bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38  -1 ${i}_R1.fq.gz | samtools view -bS - >${i%}.bam;
  
  done
  
  ```
  
  So if i use run this
  
  ```
  for i in $(ls *.fq*.gz | cut -f1-5 -d"_" | uniq);do echo $i; done
  
  ```
  
  I get files as such
  
  ```
  SO_8543_INPUT_1_ablt
  SO_8543_INPUT_2_ablt
  SO_8543_INPUT_3_ablt
  SO_8543_INPUT_4_ablt
  SO_8543_IP_1_ablt
  SO_8543_IP_2_ablt
  SO_8543_IP_3_ablt
  SO_8543_IP_4_ablt
  SO_8543_IP_5_ablt
  SO_8543_IP_6_ablt
  SO_8543_IP_7_ablt
  SO_8543_IP_8_ablt
  
  ```
  
  The part of the file name that is consistent is `_R1.fq.gz` which goes after the input argument. As these are single end files so only 1 im giving one end as input.
  
  Error message
  
  ```
  ***
  Error: Must specify at least one read input with -U/-1/-2
  (ERR): bowtie2-align exited with value 1
  ./Chip_SEQ.sh: line 25: syntax error near unexpected token `done'
  ./Chip_SEQ.sh: line 25: `done'
  
  ```
  
  it looks like its not able to recognize as input .Any help or suggestion would be highly appreciated
  
  **Tags:** shell
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-08. Score: 6 | Accepted answer
  
  You have an extra `done`, instead you want:
  
  ```
  for i in $(ls *.fq*.gz | cut -f1-5 -d"_" | uniq); do
      echo $i
      bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38 -1 ${i}_R1.fq.gz | samtools view -bS - >${i}.bam
  done
  
  ```
  
  As an aside, I'm not a big fan of piping `ls` to cut and then uniq. Why not instead:
  
  ```
  for f in *.fq.gz; do
      sampleName=${f%%_R1.fq.gz}
      echo $sampleName
      bowtie2 -q ...
  done
  
  ```
  
  That will do the same thing, but (A) it's clearer what's going on and (B) it can be reused if your sample names change. Also, stop writing bash scripts and use snakemake or something like that instead, it scales much better and is less error-prone.
  
  > **Comments:**
  > **username_0**: now after looking at my script number of times i figured out the "done" is extra .let me try your code. My shell scripting is very rudimentary so i use which i know is safe to save my self from screwing up .  
  > **username_0**: i ran your first code the but i still get this error "Error: Must specify at least one read input with -U/-1/-2 (ERR): bowtie2-align exited with value 1" its  not able to recognize the input file, not sure what's wrong .  
  > **username_1**: Try `-U` instead of `-1` in the bowtie2 command.  
  > **username_0**: yes that was the mistake i was doing thank you very much  
  > **username_2**: @krushnachChandra please read [this link](https://mywiki.wooledge.org/ParsingLs) to understand why you should basically _never_ try to parse the output of `ls`.  
  > **username_3**: @username_2 I'd not say "never parse ls", I'd say "Don't parse ls unless you know exactly what you're doing, especially the filenames involved". I do a lot of `for f in $(/bin/ls -1 <glob> | downstream_operations)` because I'm 100% certain that each value of `$f` plays well with the other parts in the code.  
  > **username_2**: @username_3 well, parsing ls is asking for trouble and just unnecessary. `for f in <glob>` does exactly the same thing but without calling an external process and without running any risk of failure because of file names. So why use a riskier and less efficient approach when it doesn't provide any benefit at all?  
  > **username_3**: It's really useful when the `$()` feeding the loop involves a `uniq`, for example. It can't always be avoided. Whenever possible, use a glob. But don't have a hard constraint against `$(/bin/ls | downstream_stuff)`  
  > **username_0**: thats a life saver link  
  
  ---
  
  