Answers.Body: <p>I'd recommend PCA, tSNE the PCA (first)+ tSNE (second).</p>
  <p>The sequence data - one hot key encoding is a good idea.</p>
  <pre><code>import pandas as pd
  datadict = {'UTR':[.....],'Sequence':['AGAA','ACAG' ...], 'Optimal Secondary Structure':[....]}
  df = pd.DataFrame(datadict)
  df_onehot = pd.get_dummies(df['Sequence'])
  df = pd.concat([df, df_onehot], axis=1)     # add the one hot encoded column to master dataframe
  df.drop(...) # remove the original 'Sequence' column
  </code></pre>
  <hr />
  <p>The other issue is vectorising the &quot;<strong>dots</strong>&quot;. This is really a separate question, here's what I'd do ...</p>
  <p>For optimal secondary structure, maybe simply count the dots?</p>
  <pre><code>count = df_series.str.count(&quot;.&quot;)
  </code></pre>
  <p>This only works on a series so you'll need to do something like <code>df_list = df['Dots column'].to_list()</code> and then <code>df_series = pd.Series(df_list)</code>. Finally the <code>count</code> will need to be <code>concat</code> back onto the <code>df</code> and the original <code>Dots</code> deleted.</p>
  <p>The thing I'm not clear about is your 'instances' that could be 'UTR_info' thats the stuff you are calling you data or &quot;features&quot;.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-09T23:48:03.620Z
Answers.Text: Got it, but how does pca and tsne get me closer to actually identifying structural/sequence patterns present in the created clusters?
Answers.UserDisplayName: joshua213p
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-09T23:58:01.273Z
Answers.Text: Thanks. *You can now upvote BTW*. This is about the fundamental design of your experiment. If post another question I'll sketch an outline about what unsupervised learning achieves. I don't exactly know your project aims (I can guess) but I know what unsupervised learning delivers.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-09T23:06:02.383Z
Answers.LastActivityDate: 2023-08-09T23:26:16.923Z
Answers.LastEditDate: 2023-08-09T23:26:16.923Z
Answers.UserDisplayName: M__
Body: <p>Reading through <a href="https://www.datacamp.com/tutorial/introduction-t-sne" rel="nofollow noreferrer">this</a> t-sne blogpost and I'm a little unclear on how i'd use either of these algorithms here for my goals. i get it'd cluster my samples based on similarity in terms of first and second principal components/t-SNEs, but what could i then use identify structural or sequence patterns within each cluster? please let me know if I'm missing something - thanks!</p>
  <p>I'm also curious as to how I would apply PCA or t-SNE on my data considering its non-numerical nature. I was thinking of applying one-hot encoding, but is there a generally more suggested approach? I've attached a screenshot of a few of my dataset rows for context (for the sake of having the table fit, I chose rows with simpler structures/zero MFE values, but most rows have complex structures and non-zero MFE values). Thanks.</p>
  <p><a href="https://i.sstatic.net/GbuKU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/GbuKU.png" alt="enter image description here" /></a></p>
  <hr />
  <p>This is based on my previous post:</p>
  <blockquote>
  <p>I have a bunch of RNA sequences (and their optimal secondary structures) and their corresponding energy values (measured by mean free energy) and I'm trying to find a way to identify features (patterns in their structures or sequences) common between samples of similar energy values. the overall goal is to try and identify sequence or structural elements in the RNA samples that tend to correlate with low mean free energy values. would be grateful if anyone could recommend me algorithms to look into using - I'm assuming this would be an unsupervised learning project and I only have experience with supervised stuff, but I'm looking into PCA right now and not sure if that'd be useful. should I be looking into something else?</p>
  </blockquote>
  <blockquote>
  <p>Intuitively I'm imagining it as like a clustering + feature extraction problem where I have a bunch of dots, each representing an RNA sample, and then the axis represents energy so the dots could be clustered in energy value similarity and then within each energy value cluster patterns/relationships could be found between the sequences and structures of the samples. but not sure if an ML algo exists to handle this and PCA seems like not what I'm looking for because the axes would be principal components and not energy...</p>
  </blockquote>
  
Tags: <rna><clustering><unsupervised-learning>
Title: Applying PCA or tSNE on non-numerical data
ThreadText: # Applying PCA or tSNE on non-numerical data
  
  (Asked by: username_0 on 2023-08-09)
  
  Reading through [this](https://www.datacamp.com/tutorial/introduction-t-sne) t-sne blogpost and I'm a little unclear on how i'd use either of these algorithms here for my goals. i get it'd cluster my samples based on similarity in terms of first and second principal components/t-SNEs, but what could i then use identify structural or sequence patterns within each cluster? please let me know if I'm missing something - thanks!
  
  I'm also curious as to how I would apply PCA or t-SNE on my data considering its non-numerical nature. I was thinking of applying one-hot encoding, but is there a generally more suggested approach? I've attached a screenshot of a few of my dataset rows for context (for the sake of having the table fit, I chose rows with simpler structures/zero MFE values, but most rows have complex structures and non-zero MFE values). Thanks.
  
  [![enter image description here](https://i.sstatic.net/GbuKU.png)](https://i.sstatic.net/GbuKU.png)
  
  ---
  
  This is based on my previous post:
  
  > I have a bunch of RNA sequences (and their optimal secondary structures) and their corresponding energy values (measured by mean free energy) and I'm trying to find a way to identify features (patterns in their structures or sequences) common between samples of similar energy values. the overall goal is to try and identify sequence or structural elements in the RNA samples that tend to correlate with low mean free energy values. would be grateful if anyone could recommend me algorithms to look into using - I'm assuming this would be an unsupervised learning project and I only have experience with supervised stuff, but I'm looking into PCA right now and not sure if that'd be useful. should I be looking into something else?
  
  > Intuitively I'm imagining it as like a clustering + feature extraction problem where I have a bunch of dots, each representing an RNA sample, and then the axis represents energy so the dots could be clustered in energy value similarity and then within each energy value cluster patterns/relationships could be found between the sequences and structures of the samples. but not sure if an ML algo exists to handle this and PCA seems like not what I'm looking for because the axes would be principal components and not energy...
  
  **Tags:** rna, clustering, unsupervised-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-09. Score: 1 | Accepted answer
  
  I'd recommend PCA, tSNE the PCA (first)+ tSNE (second).
  
  The sequence data - one hot key encoding is a good idea.
  
  ```
  import pandas as pd
  datadict = {'UTR':[.....],'Sequence':['AGAA','ACAG' ...], 'Optimal Secondary Structure':[....]}
  df = pd.DataFrame(datadict)
  df_onehot = pd.get_dummies(df['Sequence'])
  df = pd.concat([df, df_onehot], axis=1)     # add the one hot encoded column to master dataframe
  df.drop(...) # remove the original 'Sequence' column
  
  ```
  
  ---
  
  The other issue is vectorising the "**dots**". This is really a separate question, here's what I'd do ...
  
  For optimal secondary structure, maybe simply count the dots?
  
  ```
  count = df_series.str.count(".")
  
  ```
  
  This only works on a series so you'll need to do something like `df_list = df['Dots column'].to_list()` and then `df_series = pd.Series(df_list)`. Finally the `count` will need to be `concat` back onto the `df` and the original `Dots` deleted.
  
  The thing I'm not clear about is your 'instances' that could be 'UTR\_info' thats the stuff you are calling you data or "features".
  
  > **Comments:**
  > **username_0**: Got it, but how does pca and tsne get me closer to actually identifying structural/sequence patterns present in the created clusters?  
  > **username_1**: Thanks. *You can now upvote BTW*. This is about the fundamental design of your experiment. If post another question I'll sketch an outline about what unsupervised learning achieves. I don't exactly know your project aims (I can guess) but I know what unsupervised learning delivers.  
  
  ---
  
  