These are chat archives for nextflow-io/nextflow
-resumebehaviour. I am using a GATK GenomicsDBImport process in my NF pipeline, split by chromosomes. Even if a chromosome has finished nicely, when I re-run my pipeline,
-resumedoes not work as intended and it re-runs the process. I am using NF and
-resumefor other pipelines and others processes with no trouble at all. Where could I look to investigate what falsely triggers the re-run?
process GenomicsDBImport { cpus 1 memory '72 GB' time '12h' tag { chr } input: each chr from chromosomes_ch file (gvcf) from gvcf_ch.toList() file (gvcf_idx) from gvcf_idx_ch.toList() output: set chr, file ("${params.cohort}.${chr}.tar") into gendb_ch script: """ ${GATK} GenomicsDBImport --java-options "-Xmx24g -Xms24g" \ ${gvcf.collect { "-V $it " }.join()} \ -L ${chr} \ --genomicsdb-workspace-path ${params.cohort}.${chr} tar -cf ${params.cohort}.${chr}.tar ${params.cohort}.${chr} """ }
toList()is randomizing the input?
file (gvcf) from gvcf_ch.toList() file (gvcf_idx) from gvcf_idx_ch.toList()
-resume, chrY skipped as intended! Thanks Paolo!!
binas
afterScript
/usr/share/univage/soldierantcluster/spool/node-hp0209/job_scripts/26382096: line 103: fixQcml.sh: command not found
I have a problem with
-resume and adding the
lenient directive doesn't help
The process that always reruns is the trimming process (see below). Does the fact that it has a stdout output as an input make a difference?
process determine_min_read_length { tag { pair_id } input: set pair_id, file(file_pair) from raw_fastqs_for_length_assessment output: stdout min_read_length """ seqtk sample -s123 ${file_pair[0]} 1000 | printf "%.0f\n" \$(awk 'NR%4==2{sum+=length(\$0)}END{print sum/(NR/4)/3}') """ } // Trimming process trimming { tag { pair_id } input: val(min_len) from min_read_length file('adapter_file.fas') from adapter_file set pair_id, file(file_pair) from raw_fastqs_for_trimming .....
raw_fastqs_for_length_assessmentdefined ?
Channel .fromFilePairs( fastqs ) .ifEmpty { error "Cannot find any reads matching: ${fastqs}" } .set { raw_fastqs } // duplicate raw fastq channel raw_fastqs.into { raw_fastqs_for_trimming; raw_fastqs_for_length_assessment; raw_fastqs_for_genome_size_estimation; raw_fastqs_for_read_counting}
Does the fact that it has a stdout output as an input make a difference?
val(min_len) from min_read_length
val(min_len) from '100'
def Correspondence = [] Correspondence["MS2specCount"]["shotgun"] = "0000007"
Oct-08 15:40:29.392 [main] DEBUG nextflow.Session - Session aborted -- Cause: argument type mismatch Oct-08 15:40:29.582 [main] ERROR nextflow.cli.Launcher - @unknown java.lang.IllegalArgumentException: argument type mismatch
def Correspondence = [:] Correspondence["MS2specCount"] = ["shotgun": "0000007"]
ERROR ~ You tried to use a map entry for an index operation, this is not allowed. Maybe something should be set in parentheses or a comma is missing? @ line 135, column 32. Correspondence["MS2specCount"]["shotgun" : "0000007"] ^ 1 error
Yeah, that is the English version, but that is not how it used here or in Austria
Bwas to hard to pronounce, followed by the
r
If I run this -resume works correctly. Somehow stdout is breaking thisIf I run this -resume works correctly. Somehow stdout is breaking this
val(min_len) from '100'
WARN: Process configuration syntax $processName has been deprecated
output: set sample_id, internal_code, "shotgun_qc4l_cid", checksum, file("${sample_id}.featureXML") into shot_qc4l_cid_featureXMLfiles_for_calc_peptide_area
Missing output file(s) `shotgun_qc4l_cid` expected by process `shotgun_qc4l_cid (9d9d9d1b-9d9d-4f1a-9d27-9d2f7635059d_QC03_b6f96ae8248181b823f5a720c09445cd)` -- Error is ignored
val("shotgun_qc4l_cid")
-dump-hashesand compare two executions