 the UnifiedGenotyper Queue script
Hi,
I have several questions about the function "UnifiedGenotyper" and its corresponding Qscript.
- I first test this function on a single chromosome of my chicken sample (bam file is about 800M) to call SNPs, and set the ploidy value to 30. It took me half a day to finish. Is it normal that it took so long for a single chromosome?) } }
Best Answer
sample_ploidy is of the wrong type here. It needs to be
Option[Int]instead of
List[String]. There's an additional tricky detail so here's the whole section:
import org.broadinstitute.gatk.utils.commandline.ClassType @Argument(doc="Ploidy (number of chromosomes) per sample.", shortName="ploidy") @ClassType(classOf[Int]) var sample_ploidy: Option[Int] = None
Answers
Hmm, I'm not sure what's going on here -- can you try using a different name (like samplePloidy) for defining your ploidy variable? I wonder if it's a naming collision of some kind.
Note that Queue scatter-gather depends on having a job scheduler set up to dispatch jobs to different nodes. Do you have a job scheduler set up.
sample_ploidy is of the wrong type here. It needs to be
Option[Int]instead of
List[String]. There's an additional tricky detail so here's the whole section:
Oh nice catch @thibault.
Hi,
Thank you both for your reply!
@thibault I tried your way. The script runs smoothly, but still the output is based on sample_ploidy=2.
As you suggested, I added the line "import org.broadinstitute.gatk.utils.commandline.ClassType" as follows:
package org.broadinstitute.gatk.queue.qscripts.examples
import org.broadinstitute.gatk.queue.QScript
import org.broadinstitute.gatk.queue.extensions.gatk._
import org.broadinstitute.gatk.utils.commandline.ClassType
class TestUnifiedGenotyper extends QScript {
qscript =>
.......
Then define the argument ploidy as integer as
"
@Argument(doc="Ploidy (number of chromosomes) per sample.", shortName="ploidy")
@ClassType(classOf[Int])
var sample_ploidy: Option[Int] = None
"
And I ran with the same bash command as I posted above, it still did not respond to my ploidy value assignment.
Please help me in pointing out where I did wrong! Thanks a lot!
@Geraldine_VdAuwera
About the job schedular, the server I work on has a system called "The Slurm Workload Manager". Is this a type of job schedular that scatter-gather requires?
I tried different values to the two arguments "this.memoryLimit" and "genotyper.scatterCount".
I did not find any obvious difference in applying different values to " this.memoryLimit".
As for "genotyper.scatterCount", I do find that the higher value I assigned to it, the more "jobs" it can be splited into.
However, no matter what value it had assigned to, the "jobs" seem to run one after the other, rather than parallel.
The following information were extracted from the log file,
"INFO 09:27:08,210 QGraph - 18 Pend, 1 Run, 0 Fail, 0 Done
.......
INFO 09:28:32,013 QGraph - 17 Pend, 1 Run, 0 Fail, 1 Done
.......
INFO 09:29:21,840 QGraph - 16 Pend, 1 Run, 0 Fail, 2 Done
"
Last, based on your experience, I need 12hours on 16 cores (with the help of options "-nct" and "-nt") to complete variant calling on a 800M bam file, is it a normal case?
Thanks a lot for your help and time!
Queue does not run anything in parallel by itself. You need to tell it to submit jobs to a job scheduler. SLURM is an example of this, so it sounds like you're in luck! We don't support SLURM specifically (and we don't run it) but here are some user-contributed suggestions:
Please post the current version of your script here. When I tested the approach I suggested to you, I did see the ploidy correctly applied to subtasks. Maybe something else is wrong?
Thank you @thibault for your answer! It works and I need to use the commandline "genotyper.sample_ploidy = qscript.sample_ploidy" in the def script(){...} as well.
And thank you for pointing me to the thread that answered my other question!
Thank you @thibault and @Geraldine_VdAuwera both for your help and time!