Answers.Body: <p>Another option is to use <code>list.files</code> and <code>grep</code> to select only the <code>.txt</code> in your directory. A <code>for</code> loop can read-aggregate and save the files one at the time.</p>
  
  <pre><code>all.files &lt;- list.files("/location folder")
  txt.files &lt;- grep(".txt",all.files,value=T)
  
  for(i in txt.files){
    d &lt;- read.table(paste("/location folder",i,sep=""),header=F)
    x &lt;- aggregate(d[, 2], list(d$V3), median)
    write.table(x, paste0(i,".output.txt"), quote=FALSE, row.names=TRUE)
  }
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T15:07:02.573Z
Answers.Text: Why not just `list.files(pattern = "*.txt")`? The `grep` is not required.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T15:07:52.017Z
Answers.Text: And looping when vectorized operations are available is de-optimization.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T15:10:50.797Z
Answers.Text: And there's some inconsistent usage: `paste(..., sep="")` on line 5 and `paste0()` on line 7.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T15:21:42.803Z
Answers.Text: considering the nature of the question I thought that it was more important to show possibilities rather than one single point of view. `paste` and `paste0` achieve the same but in different ways (e.g. `sep=""`).  I tend to split commands involving `list.files` and `grep` as it makes easier to read and to reuse. The content of `all.files` is hard to miss. Finally the usage of the `apply`family is not easy to grasp for new user, while loops are much intuitive. Furthermore, `apply` is not always better than a nice `for`, especially in an example like this.
Answers.UserDisplayName: fra
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T15:32:25.807Z
Answers.Text: Showing possibilities is great! However, I don't think a loop would perform as well as or better than vectorized operations. While a loop is easier for people that know basic programming, the `apply` family solves a lot of problems, especially when there is a learning opportunity such as this one, where the task is well defined and how apply works can be easily understood.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-03T14:29:13.357Z
Answers.Text: Just encountered this particular gotcha, so we might have to be careful about using `T`/`F` instead of `TRUE`/`FALSE` as well: https://www.johndcook.com/blog/r_language_for_programmers/#assignment
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T14:21:56.647Z
Answers.LastActivityDate: 2019-07-02T14:21:56.647Z
Answers.UserDisplayName: fra
Answers.Body: <p>Please do not use <code>setwd</code> in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.</p>
  
  <p>Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a <code>list.files(pattern = "&lt;pattern_here&gt;")</code>. Read this file as a <code>data.frame</code> and use it with an <code>apply</code>, so a function processes the df row-by-row. Let's say the file (named <code>metadata.txt</code>) is:</p>
  
  <pre><code>in_file out_file
  tgf.txt tgf.output.txt
  abc.txt abc.filtered.txt
  xyz.txt xyz.randomname.txt
  </code></pre>
  
  <p>Define a function called <code>process_row()</code> in the Rscript:</p>
  
  <pre><code>process_row &lt;- function(r) {
      d &lt;- read.table(r[[1]], header = FALSE);
      x &lt;- aggregate(d[, 2], list(d$V3), median);
      write.table(x, r[[2]], quote=FALSE, row.names=TRUE);
  }
  </code></pre>
  
  <p>And do the actual work:</p>
  
  <pre><code>meta_data &lt;- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);
  apply(meta_data, 1, process_row(r))
  # You don't need a q() in an Rscript because it exits after the last line of code anyway.
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T09:57:28.590Z
Answers.Text: I tried the above script but it is giving me following error
  
  Error in read.table(r[[1]], header = FALSE) : 
    'file' must be a character string or connection
  Called from: read.table(r[[1]], header = FALSE)
Answers.UserDisplayName: Megha
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-02T13:40:39.020Z
Answers.Text: What is the content of `./metadata.txt`, which is in the working directory where you're running R?
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-03T14:45:50.610Z
Answers.Text: It worked. Thank you
Answers.UserDisplayName: Megha
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-03T17:42:07.027Z
Answers.Text: Please accept the answer if it worked. Thank you
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-01T16:14:38.640Z
Answers.LastActivityDate: 2019-07-02T15:11:33.410Z
Answers.LastEditDate: 2019-07-02T15:11:33.410Z
Answers.UserDisplayName: Ram RS
Body: <p>I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files</p>
  
  <pre><code> #!/usr/bin/R
   setwd("/location folder")
  
   d &lt;- read.table("tgf.txt", header=FALSE)
  
   x &lt;- aggregate(d[, 2], list(d$V3), median)
  
   write.table(x, "tgf.output.txt", quote=FALSE, row.names=TRUE)
  
   q()
  </code></pre>
  
Tags: <r>
Title: Run Rscript for multiple samples (files)
ThreadText: # Run Rscript for multiple samples (files)
  
  (Asked by: username_0 on 2019-07-01)
  
  I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files
  
  ```
   #!/usr/bin/R
   setwd("/location folder")
  
   d <- read.table("tgf.txt", header=FALSE)
  
   x <- aggregate(d[, 2], list(d$V3), median)
  
   write.table(x, "tgf.output.txt", quote=FALSE, row.names=TRUE)
  
   q()
  
  ```
  
  **Tags:** r
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-07-02. Score: 1 | Accepted answer
  
  Another option is to use `list.files` and `grep` to select only the `.txt` in your directory. A `for` loop can read-aggregate and save the files one at the time.
  
  ```
  all.files <- list.files("/location folder")
  txt.files <- grep(".txt",all.files,value=T)
  
  for(i in txt.files){
    d <- read.table(paste("/location folder",i,sep=""),header=F)
    x <- aggregate(d[, 2], list(d$V3), median)
    write.table(x, paste0(i,".output.txt"), quote=FALSE, row.names=TRUE)
  }
  
  ```
  
  > **Comments:**
  > **username_2**: Why not just `list.files(pattern = "*.txt")`? The `grep` is not required.  
  > **username_2**: And looping when vectorized operations are available is de-optimization.  
  > **username_2**: And there's some inconsistent usage: `paste(..., sep="")` on line 5 and `paste0()` on line 7.  
  > **username_1**: considering the nature of the question I thought that it was more important to show possibilities rather than one single point of view. `paste` and `paste0` achieve the same but in different ways (e.g. `sep=""`).  I tend to split commands involving `list.files` and `grep` as it makes easier to read and to reuse. The content of `all.files` is hard to miss. Finally the usage of the `apply`family is not easy to grasp for new user, while loops are much intuitive. Furthermore, `apply` is not always better than a nice `for`, especially in an example like this.  
  > **username_2**: Showing possibilities is great! However, I don't think a loop would perform as well as or better than vectorized operations. While a loop is easier for people that know basic programming, the `apply` family solves a lot of problems, especially when there is a learning opportunity such as this one, where the task is well defined and how apply works can be easily understood.  
  > **username_2**: Just encountered this particular gotcha, so we might have to be careful about using `T`/`F` instead of `TRUE`/`FALSE` as well: https://www.johndcook.com/blog/r_language_for_programmers/#assignment  
  
  ---
  
  ### Answer by username_2 on 2019-07-01. Score: 3
  
  Please do not use `setwd` in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.
  
  Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a `list.files(pattern = "<pattern_here>")`. Read this file as a `data.frame` and use it with an `apply`, so a function processes the df row-by-row. Let's say the file (named `metadata.txt`) is:
  
  ```
  in_file out_file
  tgf.txt tgf.output.txt
  abc.txt abc.filtered.txt
  xyz.txt xyz.randomname.txt
  
  ```
  
  Define a function called `process_row()` in the Rscript:
  
  ```
  process_row <- function(r) {
      d <- read.table(r[[1]], header = FALSE);
      x <- aggregate(d[, 2], list(d$V3), median);
      write.table(x, r[[2]], quote=FALSE, row.names=TRUE);
  }
  
  ```
  
  And do the actual work:
  
  ```
  meta_data <- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);
  apply(meta_data, 1, process_row(r))
  # You don't need a q() in an Rscript because it exits after the last line of code anyway.
  
  ```
  
  > **Comments:**
  > **username_0**: I tried the above script but it is giving me following error  Error in read.table(r[[1]], header = FALSE) :    'file' must be a character string or connection Called from: read.table(r[[1]], header = FALSE)  
  > **username_2**: What is the content of `./metadata.txt`, which is in the working directory where you're running R?  
  > **username_0**: It worked. Thank you  
  > **username_2**: Please accept the answer if it worked. Thank you  
  
  ---
  
  