Answers.Body: <p>If <code>fasterq-dump</code> can't read from a file, you can simply loop over the file yourself:</p>
  <pre><code>while read srr; do
    fasterq-dump &quot;$srr&quot; -o &quot;$srr&quot;.fastq
  done &lt; SraAccList.txt
  </code></pre>
  <p>That will iterate over your <code>SraAccList.txt</code> file and extract the fastqs for each accession. I tested with this command:</p>
  <pre><code>fasterq-dump SRR9004325 -o ff.fastq
  </code></pre>
  <p>And that created files <code>ff_1.fastq</code> and <code>ff_2.fastq</code>, so you can expect files named like <code>SRR9004325_1.fastq</code> and <code>SRR9004325_1.fastq</code> if you use the loop above.</p>
  <p>You could try and parallelize it but I doubt it is worth it since the bottleneck will always be the download speed.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-07T16:34:07.847Z
Answers.LastActivityDate: 2024-03-07T16:34:07.847Z
Answers.UserDisplayName: terdon
Answers.Body: <p>Note that I have switched from getting fastq files to first downloading SRA files and then converting them to fastq myself, as this is (in my experience), much faster (SRA files are smaller and somehow downloading fastq files directly was very slow for me).</p>
  <p>I first do this:</p>
  <p><code>prefetch {sample} -O prefetch_dir --max-size 50G</code></p>
  <p>Then</p>
  <pre><code>fasterq-dump {sra_file} --outdir fastq_raw_dir
  gzip fastq_raw_dir/{sample}_1.fastq
  gzip fastq_raw_dir/{sample}_2.fastq
  </code></pre>
  <p>I use Snakemake to parallelize.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-30T11:36:48.860Z
Answers.LastActivityDate: 2024-08-30T11:36:48.860Z
Answers.UserDisplayName: Freek
Body: <p>I have multiple SRR files to download, and their accession is in a list:</p>
  <pre><code>SRR9004325
  SRR9004326
  SRR9004327
  </code></pre>
  <p>I read the <a href="https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/" rel="nofollow noreferrer">NCBI document</a> but don't find how to achieve this using the command &quot;fastq-dump&quot;.</p>
  <p>And I noticed that if I use the command <code>prefetch</code>, I can download from the list file:</p>
  <pre><code>prefetch --option-file SraAccList.txt
  </code></pre>
  <p>So is there any parameter in &quot;fastq-dump&quot; to download accessions stored in a file?</p>
  <p>As @terdon said, it is better to use <code>fasterq-dump</code>, and the command:
  <code>fasterq-dump --option-file SraAccList.txt</code> doesn't work. When I try it, I just get the usage message:</p>
  <pre><code>$ fasterq-dump --option-file SraAccList.txt
  
  Usage:
    fasterq-dump &lt;path&gt; [options]
    fasterq-dump &lt;accession&gt; [options]
  
  Options:
    -F|--format                      format (special, fastq, default=fastq) 
    -o|--outfile                     output-file 
    -O|--outdir                      output-dir 
    -b|--bufsize                     size of file-buffer dflt=1MB 
    -c|--curcache                    size of cursor-cache dflt=10MB 
    -m|--mem                         memory limit for sorting dflt=100MB 
    -t|--temp                        where to put temp. files dflt=curr dir 
    -e|--threads                     how many thread dflt=6 
    -p|--progress                    show progress 
    -x|--details                     print details 
    -s|--split-spot                  split spots into reads 
    -S|--split-files                 write reads into different files 
    -3|--split-3                     writes single reads in special file 
    --concatenate-reads              writes whole spots into one file 
    -Z|--stdout                      print output to stdout 
    -f|--force                       force to overwrite existing file(s) 
    --skip-technical                 skip technical reads 
    --include-technical              include technical reads 
    -M|--min-read-len                filter by sequence-len 
    --table                          which seq-table to use in case of pacbio 
    -B|--bases                       filter by bases 
    -A|--append                      append to output-file 
    --fasta                          produce FASTA output 
    --fasta-unsorted                 produce FASTA output, unsorted 
    --fasta-ref-tbl                  produce FASTA output from REFERENCE tbl 
    --fasta-concat-all               concatenate all rows and produce FASTA 
    --internal-ref                   extract only internal REFERENCEs 
    --external-ref                   extract only external REFERENCEs 
    --ref-name                       extract only these REFERENCEs 
    --ref-report                     enumerate references 
    --use-name                       print name instead of seq-id 
    --seq-defline                    custom defline for sequence:  <span class="math-container">$ac=accession, 
                                 $</span>sn=spot-name,  <span class="math-container">$sg=spot-group, $</span>si=spot-id,  
                                     <span class="math-container">$ri=read-id, $</span>rl=read-length 
    --qual-defline                   custom defline for qualities:  same as 
                                     seq-defline 
    -U|--only-unaligned              process only unaligned reads 
    -a|--only-aligned                process only aligned reads 
    --disk-limit                     explicitly set disk-limit 
    --disk-limit-tmp                 explicitly set disk-limit for temp. files 
    --size-check                     switch to control: on=perform size-check 
                                     (default),  off=do not perform size-check,  
                                     only=perform size-check only 
    --ngc &lt;PATH&gt;                     PATH to ngc file 
  
    -h|--help                        Output brief explanation for the program. 
    -V|--version                     Display the version of the program then 
                                     quit. 
    -L|--log-level &lt;level&gt;           Logging level as number or enum string. One 
                                     of (fatal|sys|int|err|warn|info|debug) or 
                                     (0-6) Current/default is warn. 
    -v|--verbose                     Increase the verbosity of the program 
                                     status messages. Use multiple times for more 
                                     verbosity. Negates quiet. 
    -q|--quiet                       Turn off all status messages for the 
                                     program. Negated by verbose. 
    --option-file &lt;file&gt;             Read more options and parameters from the 
                                     file. 
  for more information visit:
     https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump
     https://github.com/ncbi/sra-tools/wiki/08.-prefetch-and-fasterq-dump
  
  fasterq-dump : 3.0.6
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-07T10:19:01.103Z
Comments.Text: I don't understand. What is wrong with `prefetch`? That's exactly what you need, right? `fastq-dump` (which I believe is obsolete, you should use `fasterq-dump` instead) is to extract fastq files from the SRR, not for downloading.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-07T13:24:56.573Z
Comments.Text: @terdon From the documentation: " You can also avoid the prefetch step and download and convert the Run in one step by entering just the Run accession without the .sra extension in your fasterq-dump or sam-dump command".  I just want to download the fastq file.
Comments.UserDisplayName: Bioinfotec
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-07T13:52:33.070Z
Comments.Text: OK, so what happens if you run `fasterq-dump --option-file SraAccList.txt`? If you look at the tool's options (run `fasterq-dump` with no arguments), it shows that the option is there.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-07T13:58:49.663Z
Comments.Text: Hmm. I tried that and it doesn't seem to work. But please add that to the question. Have you gotten the command to work correctly for a single file? If so, you could just loop over the accessions.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-07T16:32:24.240Z
Comments.Text: @terdon Yes, the options " --option-file" is there, but the description about it is "Read more options and parameters from the file". Emmm, it is a little confusing.
Comments.UserDisplayName: Bioinfotec
Tags: <data-download><ncbi>
Title: How can I download multiple SRR* files at a time when using sratoolkit's fastq-dump?
ThreadText: # How can I download multiple SRR* files at a time when using sratoolkit's fastq-dump?
  
  (Asked by: username_0 on 2024-03-07)
  
  I have multiple SRR files to download, and their accession is in a list:
  
  ```
  SRR9004325
  SRR9004326
  SRR9004327
  
  ```
  
  I read the [NCBI document](https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/) but don't find how to achieve this using the command "fastq-dump".
  
  And I noticed that if I use the command `prefetch`, I can download from the list file:
  
  ```
  prefetch --option-file SraAccList.txt
  
  ```
  
  So is there any parameter in "fastq-dump" to download accessions stored in a file?
  
  As @username_1 said, it is better to use `fasterq-dump`, and the command:
  `fasterq-dump --option-file SraAccList.txt` doesn't work. When I try it, I just get the usage message:
  
  ```
  $ fasterq-dump --option-file SraAccList.txt
  
  Usage:
    fasterq-dump <path> [options]
    fasterq-dump <accession> [options]
  
  Options:
    -F|--format                      format (special, fastq, default=fastq) 
    -o|--outfile                     output-file 
    -O|--outdir                      output-dir 
    -b|--bufsize                     size of file-buffer dflt=1MB 
    -c|--curcache                    size of cursor-cache dflt=10MB 
    -m|--mem                         memory limit for sorting dflt=100MB 
    -t|--temp                        where to put temp. files dflt=curr dir 
    -e|--threads                     how many thread dflt=6 
    -p|--progress                    show progress 
    -x|--details                     print details 
    -s|--split-spot                  split spots into reads 
    -S|--split-files                 write reads into different files 
    -3|--split-3                     writes single reads in special file 
    --concatenate-reads              writes whole spots into one file 
    -Z|--stdout                      print output to stdout 
    -f|--force                       force to overwrite existing file(s) 
    --skip-technical                 skip technical reads 
    --include-technical              include technical reads 
    -M|--min-read-len                filter by sequence-len 
    --table                          which seq-table to use in case of pacbio 
    -B|--bases                       filter by bases 
    -A|--append                      append to output-file 
    --fasta                          produce FASTA output 
    --fasta-unsorted                 produce FASTA output, unsorted 
    --fasta-ref-tbl                  produce FASTA output from REFERENCE tbl 
    --fasta-concat-all               concatenate all rows and produce FASTA 
    --internal-ref                   extract only internal REFERENCEs 
    --external-ref                   extract only external REFERENCEs 
    --ref-name                       extract only these REFERENCEs 
    --ref-report                     enumerate references 
    --use-name                       print name instead of seq-id 
    --seq-defline                    custom defline for sequence:  $ac=accession, 
                                 $sn=spot-name,  $sg=spot-group, $si=spot-id,  
                                     $ri=read-id, $rl=read-length 
    --qual-defline                   custom defline for qualities:  same as 
                                     seq-defline 
    -U|--only-unaligned              process only unaligned reads 
    -a|--only-aligned                process only aligned reads 
    --disk-limit                     explicitly set disk-limit 
    --disk-limit-tmp                 explicitly set disk-limit for temp. files 
    --size-check                     switch to control: on=perform size-check 
                                     (default),  off=do not perform size-check,  
                                     only=perform size-check only 
    --ngc <PATH>                     PATH to ngc file 
  
    -h|--help                        Output brief explanation for the program. 
    -V|--version                     Display the version of the program then 
                                     quit. 
    -L|--log-level <level>           Logging level as number or enum string. One 
                                     of (fatal|sys|int|err|warn|info|debug) or 
                                     (0-6) Current/default is warn. 
    -v|--verbose                     Increase the verbosity of the program 
                                     status messages. Use multiple times for more 
                                     verbosity. Negates quiet. 
    -q|--quiet                       Turn off all status messages for the 
                                     program. Negated by verbose. 
    --option-file <file>             Read more options and parameters from the 
                                     file. 
  for more information visit:
     https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump
     https://github.com/ncbi/sra-tools/wiki/08.-prefetch-and-fasterq-dump
  
  fasterq-dump : 3.0.6
  
  ```
  
  **Tags:** data-download, ncbi
  
  > **Comments:**
  > **username_1**: I don't understand. What is wrong with `prefetch`? That's exactly what you need, right? `fastq-dump` (which I believe is obsolete, you should use `fasterq-dump` instead) is to extract fastq files from the SRR, not for downloading.  
  > **username_0**: @username_1 From the documentation: " You can also avoid the prefetch step and download and convert the Run in one step by entering just the Run accession without the .sra extension in your fasterq-dump or sam-dump command".  I just want to download the fastq file.  
  > **username_1**: OK, so what happens if you run `fasterq-dump --option-file SraAccList.txt`? If you look at the tool's options (run `fasterq-dump` with no arguments), it shows that the option is there.  
  > **username_1**: Hmm. I tried that and it doesn't seem to work. But please add that to the question. Have you gotten the command to work correctly for a single file? If so, you could just loop over the accessions.  
  > **username_0**: @username_1 Yes, the options " --option-file" is there, but the description about it is "Read more options and parameters from the file". Emmm, it is a little confusing.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-03-07. Score: 2
  
  If `fasterq-dump` can't read from a file, you can simply loop over the file yourself:
  
  ```
  while read srr; do
    fasterq-dump "$srr" -o "$srr".fastq
  done < SraAccList.txt
  
  ```
  
  That will iterate over your `SraAccList.txt` file and extract the fastqs for each accession. I tested with this command:
  
  ```
  fasterq-dump SRR9004325 -o ff.fastq
  
  ```
  
  And that created files `ff_1.fastq` and `ff_2.fastq`, so you can expect files named like `SRR9004325_1.fastq` and `SRR9004325_1.fastq` if you use the loop above.
  
  You could try and parallelize it but I doubt it is worth it since the bottleneck will always be the download speed.
  
  
  ---
  
  ### Answer by username_2 on 2024-08-30. Score: 0
  
  Note that I have switched from getting fastq files to first downloading SRA files and then converting them to fastq myself, as this is (in my experience), much faster (SRA files are smaller and somehow downloading fastq files directly was very slow for me).
  
  I first do this:
  
  `prefetch {sample} -O prefetch_dir --max-size 50G`
  
  Then
  
  ```
  fasterq-dump {sra_file} --outdir fastq_raw_dir
  gzip fastq_raw_dir/{sample}_1.fastq
  gzip fastq_raw_dir/{sample}_2.fastq
  
  ```
  
  I use Snakemake to parallelize.
  
  
  ---
  
  