events.action: opened
events.author: phu5ion
events.masked_author: username_0
events.text: Hi Brad,
  
  I'm having problems reading paired end fastq.gz files into bcbio. I ensured that the config file follows the same format as specified in the documentation, but it looks like multiple files in a list are not supported?
  
  Trace:
  [Wed Sep 07 14:56:07 ~/storage3/hodl/Projects/180/split_reads/structural_variants/bcbio/work ] $bcbio_nextgen.py ../config/config.yaml -n 8
  [2016-09-07T07:22Z] System YAML configuration: /home/hodl/local/share/bcbio/galaxy/bcbio_system.yaml
  [2016-09-07T07:22Z] Resource requests: bwa, sambamba, samtools; memory: 2.00, 2.00; cores: 16, 16, 16
  [2016-09-07T07:22Z] Configuring 1 jobs to run, using 8 cores each with 16.1g of memory reserved for each job
  [2016-09-07T07:22Z] Timing: organize samples
  [2016-09-07T07:22Z] multiprocessing: organize_samples
  [2016-09-07T07:22Z] Using input YAML configuration: /media/storage3/hodl/Projects/180/split_reads/structural_variants/bcbio/config/config.yaml
  Traceback (most recent call last):
    File "/home/hodl/local/bin/bcbio_nextgen.py", line 4, in <module>
      __import__('pkg_resources').run_script('bcbio-nextgen==0.9.9', 'bcbio_nextgen.py')
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 726, in run_script
  
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 1484, in run_script
  
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.9-py2.7.egg-info/scripts/bcbio_nextgen.py", line 226, in <module>
      main(**kwargs)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.9-py2.7.egg-info/scripts/bcbio_nextgen.py", line 43, in main
      run_main(**kwargs)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 43, in run_main
      fc_dir, run_info_yaml)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 87, in _run_toplevel
      for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 120, in variant2pipeline
      [x[0]["description"] for x in samples]]])
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
      return run_multicore(fn, items, config, parallel=parallel)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
      for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 800, in __call__
      while self.dispatch_one_batch(iterator):
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 658, in dispatch_one_batch
      self._dispatch(tasks)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 566, in _dispatch
      job = ImmediateComputeBatch(batch)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 180, in __init__
      self.results = batch()
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in __call__
      return [func(*args, **kwargs) for func, args, kwargs in self.items]
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper
      return apply(f, *args, **kwargs)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 300, in organize_samples
      return run_info.organize(*args)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 47, in organize
      run_details = _run_info_from_yaml(dirs, run_info_yaml, config, sample_names)
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 696, in _run_info_from_yaml
      item = _normalize_files(item, dirs.get("flowcell"))
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 618, in _normalize_files
      files = [_file_to_abs(x, [os.getcwd(), fc_dir, fastq_dir]) for x in files]
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 592, in _file_to_abs
      if x is None or os.path.isabs(x):
    File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/posixpath.py", line 54, in isabs
      return s.startswith('/')
  AttributeError: 'list' object has no attribute 'startswith'
  
  My config file for 1 sample looks like this (formatting is off)
  - algorithm:
      aligner: bwa
      recalibrate: gatk
      realign: gatk
      ensemble:
        numpass: 1
      variant_regions: /home/hodl/references/bedfiles/Nimblegen_SeqCap_EZ_Exome_v3_primary.hs37.bed
      svcaller: [lumpy, manta, delly]
    analysis: variant2
    description: T180_baseline
    files:
    - [/home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R1.fastq.gz, /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R2.fastq.gz]
    genome_build: GRCh37
    metadata:
      batch: 180_baseline
      phenotype: tumor
  
  Could you help me pls? Thank you!
events.title: Cannot read input for paired reads?
events.type: issue
events.action: created
events.author: chapmanb
events.masked_author: username_1
events.text: It looks like the fastq files are formatted in your configuration as a list of lists (the `-` starts a list). You'd either want to remove the `-` and move the list up one, or do:
  ```
  files:
    - /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R1.fastq.gz, 
   - /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R2.fastq.gz]
  ```
  Hope this gets things running for you.
events.type: comment
events.action: closed
events.author: chapmanb
events.masked_author: username_1
events.type: issue