Answers.Body: <p>Converting your <code>data.frame</code> to a <code>matrix</code> (and then removing the <code>data.frame</code>) will often free up enough memory that you won't run into this. Note that a <code>matrix</code> is more memory efficient than a <code>data.frame</code> and you're requiring <code>Rtsne()</code> to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).</p>
  
  <p>For what it's worth, it's never been entirely clear to me what the interaction is between a <code>data.frame</code> and the pointer protection stack, but it's often the case that this solves this sort of error.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-12T15:02:42.103Z
Answers.Text: `data.frames` are lists of column vectors, and every one of these columns has a pointer that may need protecting. With a huge number of columns, I can easily see how this might blow the protect stack. Dense matrices are contiguous memory, they only need a single pointer.
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-12T16:23:50.280Z
Answers.Text: I'd considered that, but my reading never clarified if it's really due to pointers for the columns or something else. It's unfortunate how little is mentioned about that in the documentation.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-04T12:07:15.483Z
Answers.LastActivityDate: 2017-09-04T12:07:15.483Z
Answers.UserDisplayName: Devon Ryan
Body: <p>I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.</p>
  
  <p>Here is the code that I'm trying to run.</p>
  
  <pre><code># t-SNE implementation
  library(Rtsne)
  
  set.seed(1)
  
  for(i in 1:15){
  
    tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)
  
    colors = rainbow(length(unique(data.T$classes)))
    names(colors) = unique(data.T$classes)
    plot(tsne$Y, t='n', main="tsne")
    text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])
  
    readline(prompt="Press [enter] to continue")
  }
  </code></pre>
  
  <p>Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.</p>
  
  <p>The think here is that when I execute this script, R returns me this error:</p>
  
  <blockquote>
    <p>Error: protect(): protection stack overflow</p>
  </blockquote>
  
  <p>Should I change the <code>--max-pp-size</code> or it's a bug in Rtsne package?</p>
  
  <p>Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.</p>
  
  <p>For the configuration of the Rtsne function I read <a href="https://distill.pub/2016/misread-tsne/" rel="noreferrer">this article</a></p>
  
  <p>Any other suggestion on the implementation is welcomed.</p>
  
Tags: <r><visualization><microarray><clustering>
Title: Using the t-SNE algorithm on microarray data + an error bonus
ThreadText: # Using the t-SNE algorithm on microarray data + an error bonus
  
  (Asked by: username_0 on 2017-09-02)
  
  I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.
  
  Here is the code that I'm trying to run.
  
  ```
  # t-SNE implementation
  library(Rtsne)
  
  set.seed(1)
  
  for(i in 1:15){
  
    tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)
  
    colors = rainbow(length(unique(data.T$classes)))
    names(colors) = unique(data.T$classes)
    plot(tsne$Y, t='n', main="tsne")
    text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])
  
    readline(prompt="Press [enter] to continue")
  }
  
  ```
  
  Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.
  
  The think here is that when I execute this script, R returns me this error:
  
  > Error: protect(): protection stack overflow
  
  Should I change the `--max-pp-size` or it's a bug in Rtsne package?
  
  Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.
  
  For the configuration of the Rtsne function I read [this article](https://distill.pub/2016/misread-tsne/)
  
  Any other suggestion on the implementation is welcomed.
  
  **Tags:** r, visualization, microarray, clustering
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-09-04. Score: 13 | Accepted answer
  
  Converting your `data.frame` to a `matrix` (and then removing the `data.frame`) will often free up enough memory that you won't run into this. Note that a `matrix` is more memory efficient than a `data.frame` and you're requiring `Rtsne()` to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).
  
  For what it's worth, it's never been entirely clear to me what the interaction is between a `data.frame` and the pointer protection stack, but it's often the case that this solves this sort of error.
  
  > **Comments:**
  > **username_2**: `data.frames` are lists of column vectors, and every one of these columns has a pointer that may need protecting. With a huge number of columns, I can easily see how this might blow the protect stack. Dense matrices are contiguous memory, they only need a single pointer.  
  > **username_1**: I'd considered that, but my reading never clarified if it's really due to pointers for the columns or something else. It's unfortunate how little is mentioned about that in the documentation.  
  
  ---
  
  