Body: <p>My aim is to create an embedding with AI models and to reconstruct the original data as well as possible from the embedding.</p>
  <p>My data is the CCLE dataset with 16,383 gene expressions from 1037 cell lines.
  I created two different models:</p>
  <ul>
  <li>A Transformer model with MultiHeadAttention (MSELoss of 0.4)</li>
  <li>A simple VAE (MSELoss of 0.6)</li>
  </ul>
  <p>The 0.4 means that the value per position of the 16,383 gene expressions deviates by 0.4 from the original data.</p>
  <p>I tried a lot with hyperparameters but I can't get below this value.</p>
  <p>Do you think this can be the truth, or is there perhaps an even better way to create an embedding from which you can get the best possible reconstruction?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-09T07:47:37.717Z
Comments.Text: Could you clarify what the input and output data was, beyond "gene expression"? E.g. what input and output vectors are, what the sequence represents - are these amino acid sequences for instance, and what starting sequences are you testing with? Is your loss from training data, or separate hold-out test sequences
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-13T16:04:03.120Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Tags: <transformer><embeddings><biology>
Title: How well can gene expression values be reconstructed?
UserDisplayName: FireBlazeTail
ThreadText: # How well can gene expression values be reconstructed?
  
  (Asked by: username_0 on 2024-10-05)
  
  My aim is to create an embedding with AI models and to reconstruct the original data as well as possible from the embedding.
  
  My data is the CCLE dataset with 16,383 gene expressions from 1037 cell lines.
  I created two different models:
  
  * A Transformer model with MultiHeadAttention (MSELoss of 0.4)
  * A simple VAE (MSELoss of 0.6)
  
  The 0.4 means that the value per position of the 16,383 gene expressions deviates by 0.4 from the original data.
  
  I tried a lot with hyperparameters but I can't get below this value.
  
  Do you think this can be the truth, or is there perhaps an even better way to create an embedding from which you can get the best possible reconstruction?
  
  **Tags:** transformer, embeddings, biology
  > **Comments:**
  > **username_1**: Could you clarify what the input and output data was, beyond "gene expression"? E.g. what input and output vectors are, what the sequence represents - are these amino acid sequences for instance, and what starting sequences are you testing with? Is your loss from training data, or separate hold-out test sequences  
  > **username_2**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  
  ---
  
  ## 0 answers
  
  