Answers.Body: <p>The <code>{}</code> isn't in any way special to the shell or <code>awk</code>. It is used as a replacement by certain other programs, off the top of my head I can only think of <code>find</code> and <code>xargs</code> where it is used by convention, but it isn't a shell thing or an awk thing. Here, you are using it inside the <code>awk</code> command so it just breaks the awk's format because <code>{ }</code> introduces a command block and this one is empty.</p>
  <p>Your file name is <code>$i</code>, so if you want to make <code>foo_mapped.paf</code> into <code>awk_foo_mapped.paf.out</code>, you need <code>awk_$i.out</code>. Something like this:</p>
  <pre><code>for file in /PATH/TO/FILES/*_mapped.paf; do 
      fileName=&quot;${file##*/}&quot;
      awk '$12&gt;=20' &quot;$file&quot; &gt; awk_&quot;$fileName&quot;.out
  done
  </code></pre>
  <p>Note how I removed the <code>print $0</code>. This is because the default action for awk when something evaluates to <code>true</code> is to print the current line, so <code>awk '$12&gt;=20 {print $0 }</code> is equivalent to <code>awk '$12&gt;=20</code>.</p>
  <p>You can also use your original approach of having <code>awk</code> deal with the file name by using awk's internal <code>FILENAME</code> variable. Like this:</p>
  <pre><code>awk '$12&gt;=20{ print &gt; &quot;awk_&quot;FILENAME&quot;.out&quot; }' /PATH/TO/FILES/*_mapped.paf
  </code></pre>
  <p>Depending on the number of files, you might need to close the file handle after writing. If you get an error about too many open files, try:</p>
  <pre><code>mawk '$12&gt;=20{print &gt; &quot;awk_&quot;FILENAME&quot;.out&quot;; close(&quot;awk_&quot;FILENAME&quot;.out&quot;)}' /PATH/TO/FILES/*_mapped.paf
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-21T16:30:51.057Z
Answers.LastActivityDate: 2022-10-21T16:47:46.080Z
Answers.LastEditDate: 2022-10-21T16:47:46.080Z
Answers.UserDisplayName: terdon
Body: <p>Hi I'm trying to filter a column in a PAF file with the AWK command and write it to a new file:</p>
  <p><code>awk '$12&gt;=20 {print $0}' /PATH/FILE.paf &gt; /OUT/PATH/FITLERED_FILE.paf</code></p>
  <p>This command makes sure the 12th column has a value of at least 20 and then prints the entire contents of the file - with the filtered column - to a new file.</p>
  <p>This works for one file but I have 90+ I need to filter through. I tried using a for loop:</p>
  <p><code>for i in /PATH/TO/FILES/*_mapped.paf; do awk '$12&gt;=20 {print $0 &gt; &quot;awk_{}.out&quot;}' $i; done</code></p>
  <p>Which didn't work. And then I tried using a pipe command:</p>
  <p><code>cat /Users/rimo/Desktop/Pat_Assemblies/flye_mapped.txt | parallel &quot;awk '{$12&gt;=20 print $0}' /Users/rimo/Desktop/Pat_Assemblies/Assemblies_Flye/{} &gt; awk_{}&quot;</code></p>
  <p>Which also didn't work. I think the problem I am having is format-based but I would appreciate it if someone could steer me in the right direction!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:20:00.233Z
Comments.Text: What operating system are you using? Do you have access to GNU awk (`gawk` on some systems, just `awk` on most Linuxes except recent Ubuntu and maybe some others).
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:22:08.967Z
Comments.Text: I'm using a mac OS terminal
Comments.UserDisplayName: rimo
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:26:54.277Z
Comments.Text: OK. Do you have [`gawk`](https://macappstore.org/gawk/)? No need to install it if you don't, it would just make some things easier if you already have it.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:28:42.420Z
Comments.Text: I don't have `gawk` but I could install it if it's easier! What would you suggest as an alternative?
Comments.UserDisplayName: rimo
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:32:34.517Z
Comments.Text: You don't really need `gawk` here. I was mistakenly thinking you wanted to edit the original file which is made easier by gawk (you can use `gawk -i inplace`) and gawk won't choke if you have too many open files, but all you need to avoid that is to close the file handle after writing (see the last command in my answer), so you really don't need to worry. Just use the default awk.
Comments.UserDisplayName: terdon
Tags: <filtering><awk><parallel><command-line><paf>
Title: Filtering file with AWK and writing output to new file
ThreadText: # Filtering file with AWK and writing output to new file
  
  (Asked by: username_0 on 2022-10-21)
  
  Hi I'm trying to filter a column in a PAF file with the AWK command and write it to a new file:
  
  `awk '$12>=20 {print $0}' /PATH/FILE.paf > /OUT/PATH/FITLERED_FILE.paf`
  
  This command makes sure the 12th column has a value of at least 20 and then prints the entire contents of the file - with the filtered column - to a new file.
  
  This works for one file but I have 90+ I need to filter through. I tried using a for loop:
  
  `for i in /PATH/TO/FILES/*_mapped.paf; do awk '$12>=20 {print $0 > "awk_{}.out"}' $i; done`
  
  Which didn't work. And then I tried using a pipe command:
  
  `cat /Users/username_0/Desktop/Pat_Assemblies/flye_mapped.txt | parallel "awk '{$12>=20 print $0}' /Users/username_0/Desktop/Pat_Assemblies/Assemblies_Flye/{} > awk_{}"`
  
  Which also didn't work. I think the problem I am having is format-based but I would appreciate it if someone could steer me in the right direction!
  
  **Tags:** filtering, awk, parallel, command-line, paf
  
  > **Comments:**
  > **username_1**: What operating system are you using? Do you have access to GNU awk (`gawk` on some systems, just `awk` on most Linuxes except recent Ubuntu and maybe some others).  
  > **username_0**: I'm using a mac OS terminal  
  > **username_1**: OK. Do you have [`gawk`](https://macappstore.org/gawk/)? No need to install it if you don't, it would just make some things easier if you already have it.  
  > **username_0**: I don't have `gawk` but I could install it if it's easier! What would you suggest as an alternative?  
  > **username_1**: You don't really need `gawk` here. I was mistakenly thinking you wanted to edit the original file which is made easier by gawk (you can use `gawk -i inplace`) and gawk won't choke if you have too many open files, but all you need to avoid that is to close the file handle after writing (see the last command in my answer), so you really don't need to worry. Just use the default awk.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-10-21. Score: 3 | Accepted answer
  
  The `{}` isn't in any way special to the shell or `awk`. It is used as a replacement by certain other programs, off the top of my head I can only think of `find` and `xargs` where it is used by convention, but it isn't a shell thing or an awk thing. Here, you are using it inside the `awk` command so it just breaks the awk's format because `{ }` introduces a command block and this one is empty.
  
  Your file name is `$i`, so if you want to make `foo_mapped.paf` into `awk_foo_mapped.paf.out`, you need `awk_$i.out`. Something like this:
  
  ```
  for file in /PATH/TO/FILES/*_mapped.paf; do 
      fileName="${file##*/}"
      awk '$12>=20' "$file" > awk_"$fileName".out
  done
  
  ```
  
  Note how I removed the `print $0`. This is because the default action for awk when something evaluates to `true` is to print the current line, so `awk '$12>=20 {print $0 }` is equivalent to `awk '$12>=20`.
  
  You can also use your original approach of having `awk` deal with the file name by using awk's internal `FILENAME` variable. Like this:
  
  ```
  awk '$12>=20{ print > "awk_"FILENAME".out" }' /PATH/TO/FILES/*_mapped.paf
  
  ```
  
  Depending on the number of files, you might need to close the file handle after writing. If you get an error about too many open files, try:
  
  ```
  mawk '$12>=20{print > "awk_"FILENAME".out"; close("awk_"FILENAME".out")}' /PATH/TO/FILES/*_mapped.paf
  
  ```
  
  
  ---
  
  