Answers.Body: <p>Something that might work is a Universal Sentence Encoder. It's not quite what you asked for, as it generates encodings directly from text, not from other word vectors, but it is designed for matching related text:</p>
  <p><a href="https://i.sstatic.net/9D8AY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9D8AY.png" alt="enter image description here" /></a></p>
  <p>The researchers who developed it claim the generated &quot;sentence embeddings can be trivially used to compute sentence level semantic similarity scores that achieve excellent performance on the semantic textual similarity (STS) Benchmark&quot;. (It may not work as well on your task, of course.)</p>
  <p>It's available on the <a href="https://www.tensorflow.org/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder" rel="nofollow noreferrer">Tensorflow Hub</a>, with a demo on <a href="https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/semantic_similarity_with_tf_hub_universal_encoder.ipynb" rel="nofollow noreferrer">Colab</a>, or you can read the research paper <a href="https://arxiv.org/abs/1803.11175" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-13T23:49:10.810Z
Answers.LastActivityDate: 2022-06-13T23:59:10.190Z
Answers.LastEditDate: 2022-06-13T23:59:10.190Z
Answers.UserDisplayName: Lee Reeves
Answers.Body: <p>Welcome to the community.</p>
  <p>So basically you would like to come up with a single vector representation for each employer considering different skills that particular employer has?</p>
  <p>If that is the case, I would say it depends on what would be your goal after doing this analysis. I have seen papers where they simply take the average of these different vectors or just sum them up. Try different approaches and see if it works in your downstream analysis.</p>
  <p>You mentioned that one motivation is to identify clusters among employers. I suggest you try averaging and summing them up and analyzing your downstream task (clustering) to see how each approach works.</p>
  <p>I will give you an example: In the language of life domain (DNA, RNA, amino acid sequences), there is a <a href="https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8" rel="nofollow noreferrer">paper</a> which describes SeqVec, a tool that utilizes ELMo language model to train and provide vector representations for protein sequences.</p>
  <p>Say you have a protein sequence as follows: HMGHGHHMGKGAAKKHMALLKGGAGLCMLK</p>
  <p>Now tokenize it in 3-mers: HMG MGH GHG HGH ...</p>
  <p>Then imagine we get a vector representation of real numbers for each of these 3-mers.</p>
  <p>Then if you want to come up with a representation for the whole protein sequence, would you get the sum of all 3-mers or get the average of it or do something else? I would say, it depends on what problem you are solving and trying different approaches to see which one works in your case.</p>
  <p>Hope my answer helped you a little bit. Cheers.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-14T06:56:41.280Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-16T17:56:13.447Z
Answers.Text: I agree about the averaging! I realize it might be necessary given that the lists are of different lengths.
Answers.UserDisplayName: Love Book
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-14T04:22:42.500Z
Answers.LastActivityDate: 2022-06-14T04:22:42.500Z
Answers.UserDisplayName: saberhq
Body: <p>Basically what I want to do is to create a single vector representation of a list of skills belonging to employees at a company (one list per employee). The embedding will be a representation of an employee's &quot;profile&quot;. The motivation behind this is (among other reasons) that I want to be able to identify clusters among the employees.</p>
  <p>Assume I already have a trained FastText model (or Word2vec) that can generate good representations of the individual words in the list.</p>
  <p>My current solution is simply to add all the word embeddings in an employee's list together (without any form of normalization). But I'm very unsure about whether this is the best approach to generating a good representation of an employee's profile.</p>
  <p>The dimensions of the vectors are 300 and there are usually around 10 to 30 skills in a single list.</p>
  <p>Any help would be greatly appreciated!</p>
  <p>Example:</p>
  <p>Let's say we have an it-consulting firm where each employee has their own set of skills. Some consultants are more experienced or versatile, thus having more skills listed in their profiles. eg we have:</p>
  <pre><code> alex_skills = ['microsoft azure', 'machine learning', 'data science', 'python', 'sklearn', 'xgboost', 'nginx', 'flask', 'SHAP', 'git', 'word2vec', 'statistics', 'deep learning', 'linux','docker compose', 'pandas']
  
  carla_skills = ['devops', 'machine learning', 'deep learning', 'continuous integration', 'kubernetes', 'python','git', 'speech recognition', 'github', 'bitbucket', 'scikit-learn', 'natural language processing', 'pandas']
  
  adam_skills = ['automation', 'robotic process automation', 'banking and finance', 'process mapping', 'IAM', 'väsentlighetsanalys', 'business intelligence', 'auditor', 'requirements handling', 'risk management', 'coordinator', 'project manager', 'data visualization']
  </code></pre>
  <p>As you can see Alex and Carla are more similar and should possibly be in the same cluster, while Adam might not be.</p>
  <p>So I wan't to make a vector representation of the entire list of skills. And then I will use these vector representations in some clustering algorithm (eg HDBscan) and by some distance metric (eg. cosine distance), capture the relation between Alex and Carla.</p>
  <p>I suspect the fact that the lists have different lengths might cause problems, therefore maybe divide by the length of the list after adding?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-13T15:59:16.523Z
Comments.Text: Hi @Love Book and welcome to AI Stack Exchange! For clarification, could you define your notion of 'best' and 'good' in this question? For example, are you wondering about a specific use case after the representation/embedding is created, or are you wondering about the most standard/general way of creating the representation (or something else)? We hope to see more of you on this site!
Comments.UserDisplayName: DeepQZero
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-13T15:59:39.570Z
Comments.Text: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Comments.UserDisplayName: Community
Tags: <natural-language-processing><word-embedding><word2vec>
Title: What is the best way to create a vector representation (with fasttext) of a list of words?
UserDisplayName: Love Book
ThreadText: # What is the best way to create a vector representation (with fasttext) of a list of words?
  
  (Asked by: username_0 on 2022-06-13)
  
  Basically what I want to do is to create a single vector representation of a list of skills belonging to employees at a company (one list per employee). The embedding will be a representation of an employee's "profile". The motivation behind this is (among other reasons) that I want to be able to identify clusters among the employees.
  Assume I already have a trained FastText model (or Word2vec) that can generate good representations of the individual words in the list.
  My current solution is simply to add all the word embeddings in an employee's list together (without any form of normalization). But I'm very unsure about whether this is the best approach to generating a good representation of an employee's profile.
  The dimensions of the vectors are 300 and there are usually around 10 to 30 skills in a single list.
  Any help would be greatly appreciated!
  Example:
  Let's say we have an it-consulting firm where each employee has their own set of skills. Some consultants are more experienced or versatile, thus having more skills listed in their profiles. eg we have:
   alex_skills = ['microsoft azure', 'machine learning', 'data science', 'python', 'sklearn', 'xgboost', 'nginx', 'flask', 'SHAP', 'git', 'word2vec', 'statistics', 'deep learning', 'linux','docker compose', 'pandas']
  
  carla_skills = ['devops', 'machine learning', 'deep learning', 'continuous integration', 'kubernetes', 'python','git', 'speech recognition', 'github', 'bitbucket', 'scikit-learn', 'natural language processing', 'pandas']
  
  adam_skills = ['automation', 'robotic process automation', 'banking and finance', 'process mapping', 'IAM', 'väsentlighetsanalys', 'business intelligence', 'auditor', 'requirements handling', 'risk management', 'coordinator', 'project manager', 'data visualization']
  
  As you can see Alex and Carla are more similar and should possibly be in the same cluster, while Adam might not be.
  So I wan't to make a vector representation of the entire list of skills. And then I will use these vector representations in some clustering algorithm (eg HDBscan) and by some distance metric (eg. cosine distance), capture the relation between Alex and Carla.
  I suspect the fact that the lists have different lengths might cause problems, therefore maybe divide by the length of the list after adding?
  
  **Tags:** natural-language-processing, word-embedding, word2vec
  > **Comments:**
  > **username_1**: Hi @Love Book and welcome to AI Stack Exchange! For clarification, could you define your notion of 'best' and 'good' in this question? For example, are you wondering about a specific use case after the representation/embedding is created, or are you wondering about the most standard/general way of creating the representation (or something else)? We hope to see more of you on this site!  
  > **username_2**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2022-06-13. Score: 1
  
  Something that might work is a Universal Sentence Encoder. It's not quite what you asked for, as it generates encodings directly from text, not from other word vectors, but it is designed for matching related text:
  
  [![enter image description here](https://i.sstatic.net/9D8AY.png)](https://i.sstatic.net/9D8AY.png)
  
  The researchers who developed it claim the generated "sentence embeddings can be trivially used to compute sentence level semantic similarity scores that achieve excellent performance on the semantic textual similarity (STS) Benchmark". (It may not work as well on your task, of course.)
  
  It's available on the [Tensorflow Hub](https://www.tensorflow.org/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder), with a demo on [Colab](https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/semantic_similarity_with_tf_hub_universal_encoder.ipynb), or you can read the research paper [here](https://arxiv.org/abs/1803.11175).
  
  
  ---
  
  ### Answer by username_4 on 2022-06-14. Score: 0
  
  Welcome to the community.
  
  So basically you would like to come up with a single vector representation for each employer considering different skills that particular employer has?
  
  If that is the case, I would say it depends on what would be your goal after doing this analysis. I have seen papers where they simply take the average of these different vectors or just sum them up. Try different approaches and see if it works in your downstream analysis.
  
  You mentioned that one motivation is to identify clusters among employers. I suggest you try averaging and summing them up and analyzing your downstream task (clustering) to see how each approach works.
  
  I will give you an example: In the language of life domain (DNA, RNA, amino acid sequences), there is a [paper](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8) which describes SeqVec, a tool that utilizes ELMo language model to train and provide vector representations for protein sequences.
  
  Say you have a protein sequence as follows: HMGHGHHMGKGAAKKHMALLKGGAGLCMLK
  
  Now tokenize it in 3-mers: HMG MGH GHG HGH ...
  
  Then imagine we get a vector representation of real numbers for each of these 3-mers.
  
  Then if you want to come up with a representation for the whole protein sequence, would you get the sum of all 3-mers or get the average of it or do something else? I would say, it depends on what problem you are solving and trying different approaches to see which one works in your case.
  
  Hope my answer helped you a little bit. Cheers.
  
  > **Comments:**
  > **username_2**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  > **username_0**: I agree about the averaging! I realize it might be necessary given that the lists are of different lengths.  
  
  ---
  
  