Answers.Body: <p>My initial (naive) approach would be to</p>
  <ul>
  <li>embed the textual data to some representation, using a pre-trained text embedding network. While BERT indeed works, I'd even start off with a simpler embedding, working my way into more complicated ones. For example - a starting baseline might even be Bag Of Words</li>
  <li>play around with <span class="math-container">$k$</span>, the number of clusters (hyper-parameter) in <span class="math-container">$k$</span>-means clustering, and see how it works.</li>
  </ul>
  <p>P.S. If you have tagged postings, you can also try a simple version of supervised learning, but I might not understand what you're referring to by tagged.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-13T22:57:38.860Z
Answers.Text: I ended up using a sequential model to start, using a GloVe word embedding matrix, and RELU + Softmax layers. I achieved 68% accuracy on my first run of training, and I have a few optimizations to make to get that number higher. I only started on 20k posts (16k train, 4k test) for this accuracy so far.
  
  From what I understand, isn't clustering something that would be done before training? or is there a way to utilize that afterwards?
  
  I'm relatively new to NLP and thought I've done well so far. I appreciate the advice and will research what you recommended for my optimizations. Thank you !
Answers.UserDisplayName: Ryley Keegan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-12T08:18:29.697Z
Answers.LastActivityDate: 2023-01-12T08:18:29.697Z
Answers.UserDisplayName: Hadar Sharvit
Body: <p>I've recently embarked on a task to classify an occupation code, given a job title &amp; description. I have come across clustering, a method of grouping data into clusters of which were not previously assigned. As well as, classification to group new data into pre-defined categories.</p>
  <p>The only problem is that most of the classification methods I've seen used are for a limited number of categories (usually 2, as high as 30), and the number of occupation codes reaches into the several hundred.</p>
  <p>Is there any resources someone can point me to, or an idea of a pre-trained model to use as a foundation? Any insights into this problem (if it's even possible) would be greatly appreciated. I currently have 2M+ human-tagged job postings to work off of. I've read up on the process of freezing a BERT models architectural layers, and feeding the training data into the softmax layer, but I'm hoping someone could let me know if that is just a waste of time.</p>
  
Tags: <natural-language-processing><classification><clustering><text-classification>
Title: Pre-Trained Model for Occupational Coding
UserDisplayName: Ryley Keegan
ThreadText: # Pre-Trained Model for Occupational Coding
  
  (Asked by: username_0 on 2023-01-11)
  
  I've recently embarked on a task to classify an occupation code, given a job title & description. I have come across clustering, a method of grouping data into clusters of which were not previously assigned. As well as, classification to group new data into pre-defined categories.
  
  The only problem is that most of the classification methods I've seen used are for a limited number of categories (usually 2, as high as 30), and the number of occupation codes reaches into the several hundred.
  
  Is there any resources someone can point me to, or an idea of a pre-trained model to use as a foundation? Any insights into this problem (if it's even possible) would be greatly appreciated. I currently have 2M+ human-tagged job postings to work off of. I've read up on the process of freezing a BERT models architectural layers, and feeding the training data into the softmax layer, but I'm hoping someone could let me know if that is just a waste of time.
  
  **Tags:** natural-language-processing, classification, clustering, text-classification
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-12. Score: 1
  
  My initial (naive) approach would be to
  
  * embed the textual data to some representation, using a pre-trained text embedding network. While BERT indeed works, I'd even start off with a simpler embedding, working my way into more complicated ones. For example - a starting baseline might even be Bag Of Words
  * play around with $k$, the number of clusters (hyper-parameter) in $k$-means clustering, and see how it works.
  
  P.S. If you have tagged postings, you can also try a simple version of supervised learning, but I might not understand what you're referring to by tagged.
  
  > **Comments:**
  > **username_0**: I ended up using a sequential model to start, using a GloVe word embedding matrix, and RELU + Softmax layers. I achieved 68% accuracy on my first run of training, and I have a few optimizations to make to get that number higher. I only started on 20k posts (16k train, 4k test) for this accuracy so far.  From what I understand, isn't clustering something that would be done before training? or is there a way to utilize that afterwards?  I'm relatively new to NLP and thought I've done well so far. I appreciate the advice and will research what you recommended for my optimizations. Thank you !  
  
  ---
  
  