Answers.Body: <p>It depends what your downstream requirement is. If you want to do count based statisitcs, then you definately shouldn't just do TMM normalisation on the TPM data as count statistics only work on real count data.</p>
  
  <p>If you are intersted in doing something like clustering or dimensionality reduction, then I wouldn't use TMM, I'd probably use rLog, which includes normalisation. </p>
  
  <p>You can have a crack at converting the TPMs back to something that looks like counts, but I not sure how well its going to work. To convert them back to REAL counts, you are going to need to know: the total number UMIs in the original dataset and the precise gene length used for each gene. Its posible the data set provides this ... Otherwise you are going to have problems. </p>
  
  <p>$\frac{TPM_i}{1,000,000}$ is the fraction of all transcripts in a sample that belong to a given gene. </p>
  
  <p>If $$fL_i = \frac{length_i}{\Sigma_{j\in G} length_j} $$
  is the fraction of the transcribed genome in a gene i. Thus $fR_i = fL_i \times \frac{TPM_i}{1,000,000}$ should be the fraction of all counts that come from a gene $i$. Multiply this by the total number of reads to get the original counts. </p>
  
  <p>The total number of reads may or may not be critcal, I think this might come out in the wash of the normalization, but it might mess with the disperision estimates, I'm not sure. But it is absolutely critical that </p>
  
  <p>1) The TPM dataset was counted across exactly the same annotation as the count datasets</p>
  
  <p>2) The gene lengths used are identical.</p>
  
  <p>This could be untrue for several reasons:</p>
  
  <ul>
  <li>Different annotation or annotation version (ENSEMBL, UCSC, REFSEQ, CCDS etc).</li>
  <li>Different subset of transcripts used, or different way of combinng transcripts.</li>
  <li>Different ways of calculating "effective length", used to correct for things like which parts of the gene reads can or can't map to (reads too close to ends, unmappable regions of the gene etc). </li>
  </ul>
  
  <p>Much safer to convert the count sets to TPM, but this would preclude doing count based statisitcs (e.g. DESeq2, edgeR, voom). </p>
  
  <p>However I'd also worry that if these datasets were processed at this point differently, they are going to have been processed in different ways, and that the data is basically uncomparable. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-02T21:41:17.807Z
Answers.Text: Thank you very much for the answer. Could you give any links to python (or R) packages that implement rlog? Could I apply rlog both to UMIs and TPMs? Even if not, then it might be a choice for me to do TPM for all the samples and then rlog.
Answers.UserDisplayName: Nikita Vlasenko
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-03T06:49:29.103Z
Answers.Text: rLog is comes from the deseq2 r package. It is designed counts, but as far as I'm aware it makes no assumptions that wouldn't also hold true for TPM.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-02T15:51:57.057Z
Answers.LastActivityDate: 2018-09-02T15:51:57.057Z
Answers.UserDisplayName: Ian Sudbery
Body: <p>I want to process a plenty of <code>scRNAseq</code> datasets in <code>python</code>, and I want to run <code>TMM</code> normalization on all of them. Unfortunately some of them are already normalized with <code>TPM</code> method. Is there a way of either converting it back to <code>UMI</code>matrix, so that <code>TMM</code> normalization could be applied to it or could we just apply <code>TMM</code> directly to <code>TPM</code>-normalized matrices? Would that work? Any suggestions would be greatly appreciated.</p>
  
Tags: <python><scrnaseq><normalization><tpm><tmm>
Title: Convert TPM-normalized matrices back to UMI in python
ThreadText: # Convert TPM-normalized matrices back to UMI in python
  
  (Asked by: username_0 on 2018-09-01)
  
  I want to process a plenty of `scRNAseq` datasets in `python`, and I want to run `TMM` normalization on all of them. Unfortunately some of them are already normalized with `TPM` method. Is there a way of either converting it back to `UMI`matrix, so that `TMM` normalization could be applied to it or could we just apply `TMM` directly to `TPM`-normalized matrices? Would that work? Any suggestions would be greatly appreciated.
  
  **Tags:** python, scrnaseq, normalization, tpm, tmm
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-09-02. Score: 1
  
  It depends what your downstream requirement is. If you want to do count based statisitcs, then you definately shouldn't just do TMM normalisation on the TPM data as count statistics only work on real count data.
  
  If you are intersted in doing something like clustering or dimensionality reduction, then I wouldn't use TMM, I'd probably use rLog, which includes normalisation.
  
  You can have a crack at converting the TPMs back to something that looks like counts, but I not sure how well its going to work. To convert them back to REAL counts, you are going to need to know: the total number UMIs in the original dataset and the precise gene length used for each gene. Its posible the data set provides this ... Otherwise you are going to have problems.
  
  $\frac{TPM\_i}{1,000,000}$ is the fraction of all transcripts in a sample that belong to a given gene.
  
  If $$fL\_i = \frac{length\_i}{\Sigma\_{j\in G} length\_j} $$
  is the fraction of the transcribed genome in a gene i. Thus $fR\_i = fL\_i \times \frac{TPM\_i}{1,000,000}$ should be the fraction of all counts that come from a gene $i$. Multiply this by the total number of reads to get the original counts.
  
  The total number of reads may or may not be critcal, I think this might come out in the wash of the normalization, but it might mess with the disperision estimates, I'm not sure. But it is absolutely critical that
  
  1) The TPM dataset was counted across exactly the same annotation as the count datasets
  
  2) The gene lengths used are identical.
  
  This could be untrue for several reasons:
  
  * Different annotation or annotation version (ENSEMBL, UCSC, REFSEQ, CCDS etc).
  * Different subset of transcripts used, or different way of combinng transcripts.
  * Different ways of calculating "effective length", used to correct for things like which parts of the gene reads can or can't map to (reads too close to ends, unmappable regions of the gene etc).
  
  Much safer to convert the count sets to TPM, but this would preclude doing count based statisitcs (e.g. DESeq2, edgeR, voom).
  
  However I'd also worry that if these datasets were processed at this point differently, they are going to have been processed in different ways, and that the data is basically uncomparable.
  
  > **Comments:**
  > **username_0**: Thank you very much for the answer. Could you give any links to python (or R) packages that implement rlog? Could I apply rlog both to UMIs and TPMs? Even if not, then it might be a choice for me to do TPM for all the samples and then rlog.  
  > **username_1**: rLog is comes from the deseq2 r package. It is designed counts, but as far as I'm aware it makes no assumptions that wouldn't also hold true for TPM.  
  
  ---
  
  