Answers.Body: <p>One important part in this is, do you have any summary labels for your &quot;sentences&quot;? Otherwise it may be hard to say if your models to good summaries or not. If your data is purely numerical, it seems you are rather looking at a compression problem than a summarization problem.</p>
  <p>As your sets lack structure, RNN will generalize poorly as those models are typically ordered. Instead you should be looking at Self Attention.</p>
  <p>One approach is to view your vector sets as nodes in graphs (possibly unconnected). The technique <a href="https://paperswithcode.com/paper/self-attention-graph-pooling" rel="nofollow noreferrer">SAGPool</a> should generalize well to your case.</p>
  <p>If you don't have summary labels you can set up a Graph Auto-Encoder using the SAGPool to train representations of your sets and treat the smallest layer as your summary.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-21T12:39:03.333Z
Answers.LastActivityDate: 2024-03-21T12:39:03.333Z
Answers.UserDisplayName: Victor Björkgren
Body: <p>I'm trying to design a system to optimize over a variable-length set (like a sentence) of variable length vectors (like words). But unlike a sentence, the order of words <strong>does not matter</strong>.</p>
  <p>I'll have to make vector embeddings for my input vectors, so far is clear. And I'm familiar with RNN's power to make summaries of sentences and I'd like to make a summary of my set as well. The question is what tool is the best for this problem. Is there an alternative to RNN that can make a better summary of a variable-length set of embeddings? What is the best way to make a summary for my set?</p>
  
Tags: <recurrent-neural-networks><embeddings>
Title: Best way to create a summary of variable length set of vectors where order does not matter
UserDisplayName: hasdrubal
ThreadText: # Best way to create a summary of variable length set of vectors where order does not matter
  
  (Asked by: username_0 on 2024-03-20)
  
  I'm trying to design a system to optimize over a variable-length set (like a sentence) of variable length vectors (like words). But unlike a sentence, the order of words **does not matter**.
  
  I'll have to make vector embeddings for my input vectors, so far is clear. And I'm familiar with RNN's power to make summaries of sentences and I'd like to make a summary of my set as well. The question is what tool is the best for this problem. Is there an alternative to RNN that can make a better summary of a variable-length set of embeddings? What is the best way to make a summary for my set?
  
  **Tags:** recurrent-neural-networks, embeddings
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-03-21. Score: 0
  
  One important part in this is, do you have any summary labels for your "sentences"? Otherwise it may be hard to say if your models to good summaries or not. If your data is purely numerical, it seems you are rather looking at a compression problem than a summarization problem.
  
  As your sets lack structure, RNN will generalize poorly as those models are typically ordered. Instead you should be looking at Self Attention.
  
  One approach is to view your vector sets as nodes in graphs (possibly unconnected). The technique [SAGPool](https://paperswithcode.com/paper/self-attention-graph-pooling) should generalize well to your case.
  
  If you don't have summary labels you can set up a Graph Auto-Encoder using the SAGPool to train representations of your sets and treat the smallest layer as your summary.
  
  
  ---
  
  