Answers.Body: <p>You are using <code>${CHROMOSOME_NAMES[@]}</code> but you never define that, so when your code gets here:</p>
  <pre><code>for c in &quot;${CHROMOSOME_NAMES[@]}&quot;; do
  ...
  done
  </code></pre>
  <p>It doesn't enter the <code>for</code> loop because <code>&quot;${CHROMOSOME_NAMES[@]}&quot;</code> is an empty array. It's the same thing as doing this in a shell session:</p>
  <pre><code>$ for c in &quot;${CHROMOSOME_NAMES[@]}&quot;; do echo &quot;This is chromosome $c&quot;; done
  $ 
  </code></pre>
  <p>Just like your script, that will produce no output since there is nothing for it to loop over.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-04T18:54:11.913Z
Answers.Text: Thank you for this I will try it out seems that I removed my code `CHROMOSOME_NAMES=( $(seq 1 22) X Y)`
Answers.UserDisplayName: Indira
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-04T18:50:00.237Z
Answers.LastActivityDate: 2023-04-04T18:50:00.237Z
Answers.UserDisplayName: terdon
Body: <p>I have been struggling to get the code I wrote for running GATK g CNV on my Linux machine (Steps Here: <a href="https://gatk.broadinstitute.org/hc/en-us/articles/360035531152" rel="nofollow noreferrer">https://gatk.broadinstitute.org/hc/en-us/articles/360035531152</a> ) to work. Specifically it seems to abort a run before running the main code (I even removed my DetermineGermlineContigPloidy step to no avail). Below is my code and then the error:</p>
  <pre class="lang-sh prettyprint-override"><code>    
  ## Paths To Be Set
  
  echo &quot;My paths have been set&quot;
  
  ### Set Bam Directory:
  #### Make sure to set a seperate directory for the batch you are doing
  
  BAM_DIR=/pathToThe/Batch_I_Am_Running
  
  ### Set the path to the reference genome:
  reference=/pathToMy/primary_assembly.fa
  
  #model_dir =/path/to/my/GATK-gCNV
  
  ### Set the path to the singularity image:
  SINGULARITY_CACHEDIR=/path/to/my/GATK
  singularity_image=$SINGULARITY_CACHEDIR/gatk_latest.sif
  
  ### Set Number Of Bins
  #### Normally 100 to 200*
  
  NUM_BINS=200
  
  ### Setting the number of threads ?? Like Cores ??
  
  NUM_THREADS=20
  
  
  
  #______________________________________________________________________________________________________________________________________________________________
      
  ## Main code
  
  echo &quot;I am about to start the main code&quot;
  
  for c in &quot;${CHROMOSOME_NAMES[@]}&quot;; do
  
    # loop over the directories containing the BAM files
    for dir in &quot;${BAM_DIR}&quot;/*/; do
  
      # get the name of the directory without the path
      dir_name=&quot;<span class="math-container">$(basename "$</span>{dir}&quot;)&quot;
  
      # get the name of the BAM file without the extension
      bam_name=&quot;<span class="math-container">$(basename "$</span>{dir}&quot;/*.bam .bam)&quot;
  
      echo &quot;Processing file: $bam_name&quot;
  
      echo &quot;I am starting Step 1: Preprocessing&quot;
  
      # Step 1: Preprocessing
      singularity exec <span class="math-container">$singularity_image PreprocessIntervals \
    -R $</span>{reference} \
        -I <span class="math-container">${dir}/$</span>{bam_name}.bam \
        -L targets.interval_list \
        --bin-length 0 \
        -imr OVERLAPPING_ONLY \
        -O targets.preprocessed.interval_list
  
      echo &quot;I am starting Step 2: AnnotateIntervals with GC content&quot;
  
      # Step 2: AnnotateIntervals with GC content
      singularity exec <span class="math-container">$singularity_image AnnotateIntervals \
    -L chr$</span>{c}.interval_list \
        -I <span class="math-container">${dir}/$</span>{bam_name}.bam \
        -R <span class="math-container">${reference} \
    -imr OVERLAPPING_ONLY \
    -O chr$</span>{c}.annotated.tsv
  
      echo &quot;I am starting Step 3: CollectReadCounts&quot;
  
      # Step 3: Collect read counts for the samples
      singularity exec <span class="math-container">$singularity_image CollectReadCounts \
    -I $</span>{dir}/<span class="math-container">${bam_name}.bam \
    -L targets.preprocessed.interval_list \
    --interval-merging-rule OVERLAPPING_ONLY \
    --format TSV \
    -O $</span>{dir_name}.${bam_name}.counts.tsv
  
      echo &quot;I am starting Step 4: FilteringIntervals based on GC-content &amp; read depth&quot;
  
      # Step 4: FilterIntervals based on GC-content and read depth
      singularity exec <span class="math-container">$singularity_image FilterIntervals \
    -L chr$</span>{c}.interval_list \
        --annotated-intervals chr<span class="math-container">${c}.annotated.tsv \
    -imr OVERLAPPING_ONLY \
    "$</span>{dir}&quot;/*.counts.tsv \
        --max-panel-norm 1.5 \
        --min-panel-norm 0.3 \
        -O chr${c}.filtered.interval_list
  
      echo &quot;Not doing Step 5 DetermineGermlineContigPloidy for now&quot;
  
      # Step 5: DetermineGermlineContigPloidy (Mandatory Step)
     # singularity exec <span class="math-container">$singularity_image DetermineGermlineContigPloidy \
   # --model-parameters $</span>{model_dir}/model_parameters.txt \
       # --contig-ploidy-calls <span class="math-container">${model_dir}/contig_ploidy_calls.tsv \
   # --contig-ploidy-priors chr20XY_contig_ploidy_priors.tsv \
   # --output-prefix $</span>{model_dir}/ploidy \
       # --standardized-output \
       # <span class="math-container">$(find "$</span>{output_dir}&quot; -name &quot;*.calls.tsv&quot; -printf '%p ') \
  
  #    for file in ${input_dir}/*.tsv
  #    do
  
      # Get sample ID from filename
  #      sample=<span class="math-container">$(basename $</span>{file} .tsv)
  #      # Run DetermineGermlineContigPloidy command
  #      singularity exec <span class="math-container">${singularity_image} DetermineGermlineContigPloidy \
  #        --input $</span>{file} \
  #        --contig-ploidy-priors <span class="math-container">${contig_ploidy_priors} \
  #        --output-prefix $</span>{output_dir}/${sample}_ploidy \
  #        --standardized-output
  #    done
  
      echo &quot;Creating shards via bins&quot;
  
      # Creating shards via bins
      singularity exec <span class="math-container">$singularity_image IntervalListTools \
    SplitIntervals \
    -R $</span>{reference} \
        -L chr<span class="math-container">${c}.interval_list \
    -scatter $</span>{NUM_BINS} \
        -o <span class="math-container">${dir}/$</span>{c}_shard.interval_list
  
      echo &quot;Starting the Shard Loop that will run GermlineCNVCaller and PostprocessGermlineCNVCalls&quot;
  
      # Loop over the shards and run GermlineCNVCaller and PostprocessGermlineCNVCalls
      for shard_file in &quot;<span class="math-container">${dir}/$</span>{c}&quot;*_shard.interval_list; do
  
        echo &quot;Starting Step 6: Run GermlineCNVCaller (Mandatory Step)&quot;
  
        # Step 6: Run GermlineCNVCaller (Mandatory Step)
        singularity exec <span class="math-container">$singularity_image GermlineCNVCaller \
      --run-mode COHORT \
      -L "$</span>{shard_file}&quot; \
          -I <span class="math-container">$(find "$</span>{dir}&quot; -name &quot;*.tsv&quot; -printf '%p ') \
          --minimum-interval-median-percentile 5.0 \
          --maximum-zeros-in-sample-percentage 40.0 \
          --annotation-cutoffs &quot;QD=2.0&quot; &quot;SOR=2.0&quot; &quot;FS=10.0&quot; &quot;MQ=50.0&quot; \
          --interval-merging-rule OVERLAPPING_ONLY \
          --output-prefix <span class="math-container">$(basename "$</span>{shard_file}&quot; .interval_list)
  
        echo &quot;Starting Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)&quot;
  
        # Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)
        singularity exec <span class="math-container">$singularity_image PostprocessGermlineCNVCalls \
      --model-shard-path "$</span>{shard_file}&quot; \
          --cnv-call-path <span class="math-container">$(basename "$</span>{shard_file}&quot; .interval_list).calls.tsv \
          --output-dir postprocess-sm \
          --scatter-count ${NUM_THREADS} \
          --num-scatter-plot-columns 8 \
          --max-zeros-in-sample-percentage 40.0 \
          --annotation-cutoffs &quot;QD=2.0&quot; &quot;SOR=2.0&quot; &quot;FS=10.0&quot; &quot;MQ=50.0&quot; \
          --interval-merging-rule OVERLAPPING_ONLY \
          --minimum-contig-length 46709983 \
          --minimum-interval-median-percentile 5.0
  
      done
  
  
    done
  
  
  done
  
      #______________________________________________________________________________________________________________________________________________________________
  </code></pre>
  <p>The &quot;.out&quot; file I received after the run aborted is:</p>
  <blockquote>
  <p>My paths have been set</p>
  <p>I am about to start the main code</p>
  </blockquote>
  <p>Why would it abort so soon (I changed the number of Gs used and that did not have any effect)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-04T18:47:25.747Z
Comments.Text: Please don't dump 168 lines of code on us and expect us to debug it. You need to make this more focused: give us a _minimal_ script that still produces the error. You haven't even put in the minimal effort to remove commented lines of code here! In this case, the error is trivial, but as a general rule, you need to help us help you.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-04T19:06:54.640Z
Comments.Text: I will try. I am still getting used to coding. Shouldn't commented lines remain?
Comments.UserDisplayName: Indira
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-04T19:10:25.440Z
Comments.Text: No worries, we all start somewhere! Just remember the first rule of debugging: ***print all the things***! By far the most common source of bugs is that the developer assumes their variables have one value, but actually the variables have another value. So the first step, _always_, should be to print out the variables, and to add simple print statements like `echo "I got to here!"` at different places in your code so you can figure out exactly where the problem occurs.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-04T19:25:17.217Z
Comments.Text: That is what I was trying to do with the `echo` statements.
Comments.UserDisplayName: Indira
Tags: <gatk><cnv>
Title: GATK g CNV code Aborts Before Running Main Code
ThreadText: # GATK g CNV code Aborts Before Running Main Code
  
  (Asked by: username_0 on 2023-04-04)
  
  I have been struggling to get the code I wrote for running GATK g CNV on my Linux machine (Steps Here: <https://gatk.broadinstitute.org/hc/en-us/articles/360035531152> ) to work. Specifically it seems to abort a run before running the main code (I even removed my DetermineGermlineContigPloidy step to no avail). Below is my code and then the error:
  
  ```sh
      
  ## Paths To Be Set
  
  echo "My paths have been set"
  
  ### Set Bam Directory:
  #### Make sure to set a seperate directory for the batch you are doing
  
  BAM_DIR=/pathToThe/Batch_I_Am_Running
  
  ### Set the path to the reference genome:
  reference=/pathToMy/primary_assembly.fa
  
  #model_dir =/path/to/my/GATK-gCNV
  
  ### Set the path to the singularity image:
  SINGULARITY_CACHEDIR=/path/to/my/GATK
  singularity_image=$SINGULARITY_CACHEDIR/gatk_latest.sif
  
  ### Set Number Of Bins
  #### Normally 100 to 200*
  
  NUM_BINS=200
  
  ### Setting the number of threads ?? Like Cores ??
  
  NUM_THREADS=20
  
  #______________________________________________________________________________________________________________________________________________________________
      
  ## Main code
  
  echo "I am about to start the main code"
  
  for c in "${CHROMOSOME_NAMES[@]}"; do
  
    # loop over the directories containing the BAM files
    for dir in "${BAM_DIR}"/*/; do
  
      # get the name of the directory without the path
      dir_name="$(basename "${dir}")"
  
      # get the name of the BAM file without the extension
      bam_name="$(basename "${dir}"/*.bam .bam)"
  
      echo "Processing file: $bam_name"
  
      echo "I am starting Step 1: Preprocessing"
  
      # Step 1: Preprocessing
      singularity exec $singularity_image PreprocessIntervals \
    -R ${reference} \
        -I ${dir}/${bam_name}.bam \
        -L targets.interval_list \
        --bin-length 0 \
        -imr OVERLAPPING_ONLY \
        -O targets.preprocessed.interval_list
  
      echo "I am starting Step 2: AnnotateIntervals with GC content"
  
      # Step 2: AnnotateIntervals with GC content
      singularity exec $singularity_image AnnotateIntervals \
    -L chr${c}.interval_list \
        -I ${dir}/${bam_name}.bam \
        -R ${reference} \
    -imr OVERLAPPING_ONLY \
    -O chr${c}.annotated.tsv
  
      echo "I am starting Step 3: CollectReadCounts"
  
      # Step 3: Collect read counts for the samples
      singularity exec $singularity_image CollectReadCounts \
    -I ${dir}/${bam_name}.bam \
    -L targets.preprocessed.interval_list \
    --interval-merging-rule OVERLAPPING_ONLY \
    --format TSV \
    -O ${dir_name}.${bam_name}.counts.tsv
  
      echo "I am starting Step 4: FilteringIntervals based on GC-content & read depth"
  
      # Step 4: FilterIntervals based on GC-content and read depth
      singularity exec $singularity_image FilterIntervals \
    -L chr${c}.interval_list \
        --annotated-intervals chr${c}.annotated.tsv \
    -imr OVERLAPPING_ONLY \
    "${dir}"/*.counts.tsv \
        --max-panel-norm 1.5 \
        --min-panel-norm 0.3 \
        -O chr${c}.filtered.interval_list
  
      echo "Not doing Step 5 DetermineGermlineContigPloidy for now"
  
      # Step 5: DetermineGermlineContigPloidy (Mandatory Step)
     # singularity exec $singularity_image DetermineGermlineContigPloidy \
   # --model-parameters ${model_dir}/model_parameters.txt \
       # --contig-ploidy-calls ${model_dir}/contig_ploidy_calls.tsv \
   # --contig-ploidy-priors chr20XY_contig_ploidy_priors.tsv \
   # --output-prefix ${model_dir}/ploidy \
       # --standardized-output \
       # $(find "${output_dir}" -name "*.calls.tsv" -printf '%p ') \
  
  #    for file in ${input_dir}/*.tsv
  #    do
  
      # Get sample ID from filename
  #      sample=$(basename ${file} .tsv)
  #      # Run DetermineGermlineContigPloidy command
  #      singularity exec ${singularity_image} DetermineGermlineContigPloidy \
  #        --input ${file} \
  #        --contig-ploidy-priors ${contig_ploidy_priors} \
  #        --output-prefix ${output_dir}/${sample}_ploidy \
  #        --standardized-output
  #    done
  
      echo "Creating shards via bins"
  
      # Creating shards via bins
      singularity exec $singularity_image IntervalListTools \
    SplitIntervals \
    -R ${reference} \
        -L chr${c}.interval_list \
    -scatter ${NUM_BINS} \
        -o ${dir}/${c}_shard.interval_list
  
      echo "Starting the Shard Loop that will run GermlineCNVCaller and PostprocessGermlineCNVCalls"
  
      # Loop over the shards and run GermlineCNVCaller and PostprocessGermlineCNVCalls
      for shard_file in "${dir}/${c}"*_shard.interval_list; do
  
        echo "Starting Step 6: Run GermlineCNVCaller (Mandatory Step)"
  
        # Step 6: Run GermlineCNVCaller (Mandatory Step)
        singularity exec $singularity_image GermlineCNVCaller \
      --run-mode COHORT \
      -L "${shard_file}" \
          -I $(find "${dir}" -name "*.tsv" -printf '%p ') \
          --minimum-interval-median-percentile 5.0 \
          --maximum-zeros-in-sample-percentage 40.0 \
          --annotation-cutoffs "QD=2.0" "SOR=2.0" "FS=10.0" "MQ=50.0" \
          --interval-merging-rule OVERLAPPING_ONLY \
          --output-prefix $(basename "${shard_file}" .interval_list)
  
        echo "Starting Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)"
  
        # Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)
        singularity exec $singularity_image PostprocessGermlineCNVCalls \
      --model-shard-path "${shard_file}" \
          --cnv-call-path $(basename "${shard_file}" .interval_list).calls.tsv \
          --output-dir postprocess-sm \
          --scatter-count ${NUM_THREADS} \
          --num-scatter-plot-columns 8 \
          --max-zeros-in-sample-percentage 40.0 \
          --annotation-cutoffs "QD=2.0" "SOR=2.0" "FS=10.0" "MQ=50.0" \
          --interval-merging-rule OVERLAPPING_ONLY \
          --minimum-contig-length 46709983 \
          --minimum-interval-median-percentile 5.0
  
      done
  
    done
  
  done
  
      #______________________________________________________________________________________________________________________________________________________________
  
  ```
  
  The ".out" file I received after the run aborted is:
  
  > My paths have been set
  >
  > I am about to start the main code
  
  Why would it abort so soon (I changed the number of Gs used and that did not have any effect)?
  
  **Tags:** gatk, cnv
  
  > **Comments:**
  > **username_1**: Please don't dump 168 lines of code on us and expect us to debug it. You need to make this more focused: give us a _minimal_ script that still produces the error. You haven't even put in the minimal effort to remove commented lines of code here! In this case, the error is trivial, but as a general rule, you need to help us help you.  
  > **username_0**: I will try. I am still getting used to coding. Shouldn't commented lines remain?  
  > **username_1**: No worries, we all start somewhere! Just remember the first rule of debugging: ***print all the things***! By far the most common source of bugs is that the developer assumes their variables have one value, but actually the variables have another value. So the first step, _always_, should be to print out the variables, and to add simple print statements like `echo "I got to here!"` at different places in your code so you can figure out exactly where the problem occurs.  
  > **username_0**: That is what I was trying to do with the `echo` statements.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-04-04. Score: 1 | Accepted answer
  
  You are using `${CHROMOSOME_NAMES[@]}` but you never define that, so when your code gets here:
  
  ```
  for c in "${CHROMOSOME_NAMES[@]}"; do
  ...
  done
  
  ```
  
  It doesn't enter the `for` loop because `"${CHROMOSOME_NAMES[@]}"` is an empty array. It's the same thing as doing this in a shell session:
  
  ```
  $ for c in "${CHROMOSOME_NAMES[@]}"; do echo "This is chromosome $c"; done
  $ 
  
  ```
  
  Just like your script, that will produce no output since there is nothing for it to loop over.
  
  > **Comments:**
  > **username_0**: Thank you for this I will try it out seems that I removed my code `CHROMOSOME_NAMES=( $(seq 1 22) X Y)`  
  
  ---
  
  