Answers.Body: <p>I have found the solution to my problem. The only thing missing from the command I used was the instruction to regard the run as a ATAC/GEX step. This is done by adding the parameter <code>--filter-dual-index</code> for GEX to the command.</p>
  <p>The complete, working commands for both parts of the same data set are therefore these:</p>
  <pre><code>cellranger-arc mkfastq --id=ATAC \
                         --run=P786_AHW55LAFX5/ \
                         --csv=ATAC.csv \
                         --filter-single-index \
                         --use-bases-mask=Y50,I8n*,Y16,Y82 \
                         --output-dir ATAC/
  
  cellranger-arc mkfastq --id=GEX \
                         --run=P786_AHW55LAFX5/ \
                         --csv=GEX.csv \
                         --filter-dual-index \
                         --use-bases-mask=Y28n*,I10,Y10n*,Y82 \
                         --output-dir GEX/
  </code></pre>
  <p>the number in the <code>--use-bases-mask</code> depends on the run parameters (see RunInfo.xml file) and must be adjusted accordingly.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-30T09:25:30.377Z
Answers.LastActivityDate: 2024-04-30T09:25:30.377Z
Answers.UserDisplayName: Assa Yeroslaviz
Answers.Body: <p>I don't think that samplesheet is right.  Just make 4 entries per sample, one per lane.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-30T05:28:40.303Z
Answers.Text: I disagree. At least I don't see anything wrong with the format. Pls see my edit to my question.
Answers.UserDisplayName: Assa Yeroslaviz
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-29T23:52:14.657Z
Answers.LastActivityDate: 2024-04-29T23:52:14.657Z
Answers.UserDisplayName: swbarnes2
Body: <p>I have a multiome dataset from a 10x run I'm trying to demultiplex using <code>cellranger-arc mkfastq</code>.</p>
  <p>The demultiplexing uses the <code>cellranger-arc mkfastq</code> command with a samplesheet in either of two forms, a simple one using a csv file, or a more extensive one called IEM (Illumina Experiment Manager).</p>
  <p>Both formats throws the same error, so I don't think it is the file itself.</p>
  <p>The command I'm using to start the demultiplexing is</p>
  <pre><code>cellranger-arc mkfastq --id=P786_GEX --run=/fs/pool/pool-bcfngs/Raw_data/P786_240424_NB500982_0590_AHW55LAFX5/ --samplesheet=/fs/pool/pool-cox-projects-bioinformatics/MPIB_NGS/P786/P786-gex-samplesheet-1.0.0.csv --use-bases-mask=Y28n*,I10,Y10n*,Y82 --output-dir /fs/pool/pool-bcfngs/Raw_data/P786_GEX/
  </code></pre>
  <p>When using the simple csv file the only difference is one has to use the parameter <code>--csv</code> instead of <code>--samplesheet</code>.</p>
  <p>The samplesheet itself is a comma-seaprated value file and looks like that:</p>
  <pre><code>Lane,Sample,Index
  *,RNAseq_A,SI-TT-B1
  *,RNAseq_R,SI-TT-C1
  </code></pre>
  <p>The <code>*</code> means that the sample can be found in all four lanes. This was run on a NextSeq500 machine, which can't separate the four lanes.</p>
  <p>The error I get is this is as such:</p>
  <pre><code>$ cellranger-arc mkfastq --id=P786_GEX --run=/fs/pool/pool-bcfngs/Raw_data/P786_240424_NB500982_0590_AHW55LAFX5/ --samplesheet=/fs/pool/pool-cox-projects-bioinformatics/MPIB_NGS/P786/P786-gex-samplesheet-1.0.0.csv --use-bases-mask=Y28n*,I10,Y10n*,Y82 --output-dir /fs/pool/pool-bcfngs/Raw_data/P786_GEX/
  cellranger-arc mkfastq (cellranger-arc-2.0.2)
  Copyright (c) 2021 10x Genomics, Inc.  All rights reserved.
  -------------------------------------------------------------------------------
  
  Auto-detecting ATAC/GEX: ATAC mode - I2 is 16 bases
  Martian Runtime - v4.0.5
  Serving UI at http://hpcl9301:42121?auth=cw0ukK_kaNpv_kf2sTe_pJRF0OXf3B7gaVweU23Pjh0
  
  Running preflight checks (please wait)...
  Checking run folder...
  Checking RunInfo.xml...
  Checking system environment...
  Emitting run information...
  Checking read specification...
  Checking samplesheet specs...
  2024-04-29 13:27:53 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET
  2024-04-29 13:27:53 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET.fork0.chnk0.main
  2024-04-29 13:27:54 [runtime] (chunks_complete) ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.split
  2024-04-29 13:27:54 [runtime] (split_complete)  ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.chnk0.main
  2024-04-29 13:32:16 [runtime] (chunks_complete) ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:32:16 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.join
  2024-04-29 13:32:16 [runtime] (join_complete)   ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:32:16 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE
  2024-04-29 13:32:16 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE.fork0.split
  2024-04-29 13:32:17 [runtime] (failed)          ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE
  
  [error] Pipestance failed. Error log at:
  P786_GEX/MAKE_FASTQS_CS/MAKE_FASTQS/MERGE_FASTQS_BY_LANE_SAMPLE/fork0/split-u7b0e2f8540/_errors
  
  Log message:
  Traceback (most recent call last):
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 659, in _main
      stage.main()
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 610, in main
      lambda: self._record_result(lambda: self._module.split(args))
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 589, in _run
      cmd()
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 610, in &lt;lambda&gt;
      lambda: self._record_result(lambda: self._module.split(args))
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 593, in _record_result
      self._result = cmd()
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py&quot;, line 610, in &lt;lambda&gt;
      lambda: self._record_result(lambda: self._module.split(args))
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/mro/tenkit/stages/make_fastqs/merge_fastqs_by_lane_sample/__init__.py&quot;, line 44, in split
      iem_df = tk_sheet.file_get_iem_data_frame(args.samplesheet_path)
    File &quot;/fs/gpfs41/lv07/fileset07/home/b_cox/yeroslaviz/cellranger/cellranger-arc-2.0.2/lib/python/tenkit/samplesheet.py&quot;, line 351, in file_get_iem_data_frame
      raise ValueError(&quot;Invalid IEM samplesheet format: %s&quot; % path)
  ValueError: Invalid IEM samplesheet format: /fs/pool/pool-bcfngs/Raw_data/P786_GEX/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u7b0e2f8439/files/samplesheet.csv
  </code></pre>
  <p>I don't know wehre to look for the error. I would appreciate any help in this regards.</p>
  <p><strong>EDIT</strong></p>
  <p>After swbarnes2 comment I added this here:</p>
  <p>so, yes the samplesheet is correct. There are two different kinds of samplesheets and Illumina and 10x both recommend using the simple format of a csv file. But I did try though both of them.</p>
  <p>I also tried this one here:</p>
  <pre><code>[Data],,,,,,,,,,
  Lane,Sample_ID,Sample_Name,Sample_Plate,Sample_Well,I7_Index_ID,index,I5_Index_ID,index2,Sample_Project,Description
  1,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  2,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  3,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  4,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  1,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  2,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  3,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  4,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  </code></pre>
  <p>It gave me the same error</p>
  <p>more can be found <a href="https://kb.10xgenomics.com/hc/en-us/articles/360001672471-Should-I-use-the-IEM-or-simple-CSV-format-sample-sheet-for-demultiplexing-with-mkfastq" rel="nofollow noreferrer">here</a> or <a href="https://www.10xgenomics.com/support/software/cell-ranger/latest/analysis/inputs/cr-mkfastq" rel="nofollow noreferrer">here</a>:</p>
  
Tags: <illumina><cellranger><demultiplexing>
Title: cellranger-arc throws an error when demultiplexing
ThreadText: # cellranger-arc throws an error when demultiplexing
  
  (Asked by: username_0 on 2024-04-29)
  
  I have a multiome dataset from a 10x run I'm trying to demultiplex using `cellranger-arc mkfastq`.
  
  The demultiplexing uses the `cellranger-arc mkfastq` command with a samplesheet in either of two forms, a simple one using a csv file, or a more extensive one called IEM (Illumina Experiment Manager).
  
  Both formats throws the same error, so I don't think it is the file itself.
  
  The command I'm using to start the demultiplexing is
  
  ```
  cellranger-arc mkfastq --id=P786_GEX --run=/fs/pool/pool-bcfngs/Raw_data/P786_240424_NB500982_0590_AHW55LAFX5/ --samplesheet=/fs/pool/pool-cox-projects-bioinformatics/MPIB_NGS/P786/P786-gex-samplesheet-1.0.0.csv --use-bases-mask=Y28n*,I10,Y10n*,Y82 --output-dir /fs/pool/pool-bcfngs/Raw_data/P786_GEX/
  
  ```
  
  When using the simple csv file the only difference is one has to use the parameter `--csv` instead of `--samplesheet`.
  
  The samplesheet itself is a comma-seaprated value file and looks like that:
  
  ```
  Lane,Sample,Index
  *,RNAseq_A,SI-TT-B1
  *,RNAseq_R,SI-TT-C1
  
  ```
  
  The `*` means that the sample can be found in all four lanes. This was run on a NextSeq500 machine, which can't separate the four lanes.
  
  The error I get is this is as such:
  
  ```
  $ cellranger-arc mkfastq --id=P786_GEX --run=/fs/pool/pool-bcfngs/Raw_data/P786_240424_NB500982_0590_AHW55LAFX5/ --samplesheet=/fs/pool/pool-cox-projects-bioinformatics/MPIB_NGS/P786/P786-gex-samplesheet-1.0.0.csv --use-bases-mask=Y28n*,I10,Y10n*,Y82 --output-dir /fs/pool/pool-bcfngs/Raw_data/P786_GEX/
  cellranger-arc mkfastq (cellranger-arc-2.0.2)
  Copyright (c) 2021 10x Genomics, Inc.  All rights reserved.
  -------------------------------------------------------------------------------
  
  Auto-detecting ATAC/GEX: ATAC mode - I2 is 16 bases
  Martian Runtime - v4.0.5
  Serving UI at http://hpcl9301:42121?auth=cw0ukK_kaNpv_kf2sTe_pJRF0OXf3B7gaVweU23Pjh0
  
  Running preflight checks (please wait)...
  Checking run folder...
  Checking RunInfo.xml...
  Checking system environment...
  Emitting run information...
  Checking read specification...
  Checking samplesheet specs...
  2024-04-29 13:27:53 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET
  2024-04-29 13:27:53 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET.fork0.chnk0.main
  2024-04-29 13:27:54 [runtime] (chunks_complete) ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.PREPARE_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.split
  2024-04-29 13:27:54 [runtime] (split_complete)  ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:27:54 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.chnk0.main
  2024-04-29 13:32:16 [runtime] (chunks_complete) ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:32:16 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET.fork0.join
  2024-04-29 13:32:16 [runtime] (join_complete)   ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.BCL2FASTQ_WITH_SAMPLESHEET
  2024-04-29 13:32:16 [runtime] (ready)           ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE
  2024-04-29 13:32:16 [runtime] (run:local)       ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE.fork0.split
  2024-04-29 13:32:17 [runtime] (failed)          ID.P786_GEX.MAKE_FASTQS_CS.MAKE_FASTQS.MERGE_FASTQS_BY_LANE_SAMPLE
  
  [error] Pipestance failed. Error log at:
  P786_GEX/MAKE_FASTQS_CS/MAKE_FASTQS/MERGE_FASTQS_BY_LANE_SAMPLE/fork0/split-u7b0e2f8540/_errors
  
  Log message:
  Traceback (most recent call last):
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 659, in _main
      stage.main()
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 610, in main
      lambda: self._record_result(lambda: self._module.split(args))
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 589, in _run
      cmd()
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 610, in <lambda>
      lambda: self._record_result(lambda: self._module.split(args))
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 593, in _record_result
      self._result = cmd()
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/external/martian/adapters/python/martian_shell.py", line 610, in <lambda>
      lambda: self._record_result(lambda: self._module.split(args))
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/mro/tenkit/stages/make_fastqs/merge_fastqs_by_lane_sample/__init__.py", line 44, in split
      iem_df = tk_sheet.file_get_iem_data_frame(args.samplesheet_path)
    File "/fs/gpfs41/lv07/fileset07/home/b_cox/username_0/cellranger/cellranger-arc-2.0.2/lib/python/tenkit/samplesheet.py", line 351, in file_get_iem_data_frame
      raise ValueError("Invalid IEM samplesheet format: %s" % path)
  ValueError: Invalid IEM samplesheet format: /fs/pool/pool-bcfngs/Raw_data/P786_GEX/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u7b0e2f8439/files/samplesheet.csv
  
  ```
  
  I don't know wehre to look for the error. I would appreciate any help in this regards.
  
  **EDIT**
  
  After username_1 comment I added this here:
  
  so, yes the samplesheet is correct. There are two different kinds of samplesheets and Illumina and 10x both recommend using the simple format of a csv file. But I did try though both of them.
  
  I also tried this one here:
  
  ```
  [Data],,,,,,,,,,
  Lane,Sample_ID,Sample_Name,Sample_Plate,Sample_Well,I7_Index_ID,index,I5_Index_ID,index2,Sample_Project,Description
  1,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  2,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  3,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  4,RNAseq_A,RNAseq_A,,,SI-TT-B1,SI-TT-B1,SI-TT-B1,SI-TT-B1,P786,
  1,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  2,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  3,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  4,RNAseq_R,RNAseq_R,,,SI-TT-C1,SI-TT-C1,SI-TT-C1,SI-TT-C1,P786,
  
  ```
  
  It gave me the same error
  
  more can be found [here](https://kb.10xgenomics.com/hc/en-us/articles/360001672471-Should-I-use-the-IEM-or-simple-CSV-format-sample-sheet-for-demultiplexing-with-mkfastq) or [here](https://www.10xgenomics.com/support/software/cell-ranger/latest/analysis/inputs/cr-mkfastq):
  
  **Tags:** illumina, cellranger, demultiplexing
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2024-04-30. Score: 2
  
  I have found the solution to my problem. The only thing missing from the command I used was the instruction to regard the run as a ATAC/GEX step. This is done by adding the parameter `--filter-dual-index` for GEX to the command.
  
  The complete, working commands for both parts of the same data set are therefore these:
  
  ```
  cellranger-arc mkfastq --id=ATAC \
                         --run=P786_AHW55LAFX5/ \
                         --csv=ATAC.csv \
                         --filter-single-index \
                         --use-bases-mask=Y50,I8n*,Y16,Y82 \
                         --output-dir ATAC/
  
  cellranger-arc mkfastq --id=GEX \
                         --run=P786_AHW55LAFX5/ \
                         --csv=GEX.csv \
                         --filter-dual-index \
                         --use-bases-mask=Y28n*,I10,Y10n*,Y82 \
                         --output-dir GEX/
  
  ```
  
  the number in the `--use-bases-mask` depends on the run parameters (see RunInfo.xml file) and must be adjusted accordingly.
  
  
  ---
  
  ### Answer by username_1 on 2024-04-29. Score: -2
  
  I don't think that samplesheet is right. Just make 4 entries per sample, one per lane.
  
  > **Comments:**
  > **username_0**: I disagree. At least I don't see anything wrong with the format. Pls see my edit to my question.  
  
  ---
  
  