Body: <p>I am downloading a set of Copy Number Variation samples from the <a href="https://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/CNV_Pipeline/" rel="nofollow noreferrer">GDC data API</a>. I noticed that some cases have multiple files (aliquots) for the same case (see for example <a href="https://portal.gdc.cancer.gov/cases/a8e79b70-76c4-4d4f-8e1d-37daebf2a921" rel="nofollow noreferrer">this case</a> --- <a href="https://portal.gdc.cancer.gov/query?filters=%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.case_id%22%2C%22value%22%3A%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_category%22%2C%22value%22%3A%5B%22Copy%20Number%20Variation%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_type%22%2C%22value%22%3A%5B%22Masked%20Copy%20Number%20Segment%22%5D%7D%7D%2C%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.samples.sample_type%22%2C%22value%22%3A%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%22Tumor%22%5D%7D%7D%5D%7D%5D%7D%5D%7D%5D%7D&amp;query=cases.case_id%20in%20%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%20and%20files.data_category%20in%20%5B%22Copy%20Number%20Variation%22%5D%20and%20files.data_type%20in%20%5B%22Masked%20Copy%20Number%20Segment%22%5D%20and%20cases.samples.sample_type%20in%20%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%20%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%20%22Tumor%22%5D" rel="nofollow noreferrer">here</a> the set of tumor samples with masked CNV in which it would be necessary to choose one). I need to have one file for each case in the resulting data matrix.</p>
  
  <p>I looked through the <a href="https://github.com/NCI-GDC" rel="nofollow noreferrer">GDC repository</a> but there's no indication, the same in the <a href="https://docs.gdc.cancer.gov/Data/Introduction/" rel="nofollow noreferrer">GDC API</a>.</p>
  
  <p>I think they are samples processed in different institution or at different times. I looked at the metadata but I cannot find a good criteria to choose one of them. Here's the metadata of an example set for a specific TCGA-LUSC case:</p>
  
  <pre><code>{'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'ELBOW_p_TCGAb39_SNP_N_GenomeWideSNP_6_A07_598508.nocnv_grch38.seg.v2.txt', 'submitter_id': 'e65d2f5e-96b5-4128-814e-368d830723f5_dr12_nocnv', 'file_id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'file_size': 27971, 'id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'created_datetime': '2018-05-20T21:25:50.066318-05:00', 'md5sum': '0faad919fb8ccfaa1cbcf04990433b2c', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}, 
  {'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'WHIRR_p_TCGA_168_169_170_redo_N_GenomeWideSNP_6_B10_845144.nocnv_grch38.seg.v2.txt', 'submitter_id': '9fda454b-946a-4760-8f3b-c24bdad9dbfe_dr12_nocnv', 'file_id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'file_size': 19377, 'id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'created_datetime': '2018-05-20T21:50:19.566069-05:00', 'md5sum': 'c2a55b28417de57d00c8e3eda038ca6b', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}]
  </code></pre>
  
  <p>In particular I thought about filtering by timestamp, but they have very close timestamps which are probably the timestamp of the final upload/update, not of the sample creation.</p>
  
  <p>I noticed that one sample has the word "redo" in the filename. However, I need a methodology that can be automatized through a Python script - I cannot do it by hand.</p>
  
  <p>I finally thought about taking the sample with more data, i.e. the sample with highest file size. However, this can be a bad criteria if the sample with smallest file size has a higher quality due to some kind of filtering.</p>
  
  <p>Have you already encountered this problem and find a solution?</p>
  
  <h3>Edit</h3>
  
  <p>By looking at the <a href="https://docs.gdc.cancer.gov/Encyclopedia/pages/TCGA_Barcode/" rel="nofollow noreferrer">legacy TCGA barcode</a>, the difference is due to portions distributed on one/more plates, analyzed by the same center. The two barcodes are:</p>
  
  <ul>
  <li>TCGA-66-2791-01A-01D-<strong>0978</strong>-01</li>
  <li>TCGA-66-2791-01A-01D-<strong>1969</strong>-01</li>
  </ul>
  
  <p>In boldface the plate id. My hypothesis is that by chance, the biggest one has shorter segments (more resolution, more lines) than the smallest, but both should have the same information (assuming the two aliquot have the same CNV).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-20T17:15:23.163Z
Comments.Text: have you tried tcga biolinks package available through R?  Have a look on the documentation,  may it help to solve your problem. https://bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html
Comments.UserDisplayName: Ammar Sabir Cheema
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-20T19:16:26.350Z
Comments.Text: Thanks @AmmarSabirCheema, I didn't know it - it is certainly useful!
Comments.UserDisplayName: gc5
Tags: <api><copy-number>
Title: Multiple files for case in GDC data portal
ThreadText: # Multiple files for case in GDC data portal
  
  (Asked by: username_0 on 2018-09-19)
  
  I am downloading a set of Copy Number Variation samples from the [GDC data API](https://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/CNV_Pipeline/). I noticed that some cases have multiple files (aliquots) for the same case (see for example [this case](https://portal.gdc.cancer.gov/cases/a8e79b70-76c4-4d4f-8e1d-37daebf2a921) --- [here](https://portal.gdc.cancer.gov/query?filters=%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.case_id%22%2C%22value%22%3A%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_category%22%2C%22value%22%3A%5B%22Copy%20Number%20Variation%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_type%22%2C%22value%22%3A%5B%22Masked%20Copy%20Number%20Segment%22%5D%7D%7D%2C%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.samples.sample_type%22%2C%22value%22%3A%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%22Tumor%22%5D%7D%7D%5D%7D%5D%7D%5D%7D%5D%7D&query=cases.case_id%20in%20%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%20and%20files.data_category%20in%20%5B%22Copy%20Number%20Variation%22%5D%20and%20files.data_type%20in%20%5B%22Masked%20Copy%20Number%20Segment%22%5D%20and%20cases.samples.sample_type%20in%20%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%20%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%20%22Tumor%22%5D) the set of tumor samples with masked CNV in which it would be necessary to choose one). I need to have one file for each case in the resulting data matrix.
  
  I looked through the [GDC repository](https://github.com/NCI-GDC) but there's no indication, the same in the [GDC API](https://docs.gdc.cancer.gov/Data/Introduction/).
  
  I think they are samples processed in different institution or at different times. I looked at the metadata but I cannot find a good criteria to choose one of them. Here's the metadata of an example set for a specific TCGA-LUSC case:
  
  ```
  {'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'ELBOW_p_TCGAb39_SNP_N_GenomeWideSNP_6_A07_598508.nocnv_grch38.seg.v2.txt', 'submitter_id': 'e65d2f5e-96b5-4128-814e-368d830723f5_dr12_nocnv', 'file_id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'file_size': 27971, 'id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'created_datetime': '2018-05-20T21:25:50.066318-05:00', 'md5sum': '0faad919fb8ccfaa1cbcf04990433b2c', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}, 
  {'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'WHIRR_p_TCGA_168_169_170_redo_N_GenomeWideSNP_6_B10_845144.nocnv_grch38.seg.v2.txt', 'submitter_id': '9fda454b-946a-4760-8f3b-c24bdad9dbfe_dr12_nocnv', 'file_id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'file_size': 19377, 'id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'created_datetime': '2018-05-20T21:50:19.566069-05:00', 'md5sum': 'c2a55b28417de57d00c8e3eda038ca6b', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}]
  
  ```
  
  In particular I thought about filtering by timestamp, but they have very close timestamps which are probably the timestamp of the final upload/update, not of the sample creation.
  
  I noticed that one sample has the word "redo" in the filename. However, I need a methodology that can be automatized through a Python script - I cannot do it by hand.
  
  I finally thought about taking the sample with more data, i.e. the sample with highest file size. However, this can be a bad criteria if the sample with smallest file size has a higher quality due to some kind of filtering.
  
  Have you already encountered this problem and find a solution?
  
  ### Edit
  
  By looking at the [legacy TCGA barcode](https://docs.gdc.cancer.gov/Encyclopedia/pages/TCGA_Barcode/), the difference is due to portions distributed on one/more plates, analyzed by the same center. The two barcodes are:
  
  * TCGA-66-2791-01A-01D-**0978**-01
  * TCGA-66-2791-01A-01D-**1969**-01
  
  In boldface the plate id. My hypothesis is that by chance, the biggest one has shorter segments (more resolution, more lines) than the smallest, but both should have the same information (assuming the two aliquot have the same CNV).
  
  **Tags:** api, copy-number
  
  > **Comments:**
  > **username_1**: have you tried tcga biolinks package available through R?  Have a look on the documentation,  may it help to solve your problem. https://bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html  
  > **username_0**: Thanks @username_1, I didn't know it - it is certainly useful!  
  
  ---
  
  ## 0 answers
  
  