Answers.Body: <p>I would just use a dictionary. A simple list lookup would tell you whether it's a recognised word or not. As an added bonus you can add some basic language processing, eg to identify inflected forms without listing them in your dictionary. Or use regular expressions to recognise ID numbers. ML is not really the right tool here. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T16:38:19.783Z
Answers.Text: What do you mean with dictionary? the problem I have is I got strings such as „526//51“ which would be an ID I want to recognize. But also „A1213“ would be an ID. And unfortunately every ID just occurs once or twice. So I can‘t use stuff like tf-id
Answers.UserDisplayName: oezguensi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T17:37:15.467Z
Answers.Text: A dictionary is basically a word list. For IDs you could use regular expression matching, like /[A-Z][0-9]+/ to recognise a letter followed by one or more digits. You could even do it the other way round: use a regular expression to recognise words (basically /[a-z]+/, possibly with a hyphen included), and treat everything else as a non-word/ID. The matching words you can then check against a dictionary/list, either of words or (in order to exclude them) brand names.
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T18:54:55.617Z
Answers.Text: I would like to use the power of labels and train a supervised model. What would be the best way to create embeddings. I read about different embedding methods such as hashing.
Answers.UserDisplayName: oezguensi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-26T08:10:22.677Z
Answers.Text: It's the wrong tool.
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T08:09:13.923Z
Answers.LastActivityDate: 2019-06-25T08:09:13.923Z
Answers.UserDisplayName: Oliver Mason
Body: <p>I am quite new to text classification. </p>
  
  <p>Using EAST text detection model, I get multiple strings that aren't words and most often have no meaning. For example, IDs, brand names, etc. I would like to classify them into two groups. Which models work the best and how should I preprocess the strings?  I wanted to use Word2Vec, but I think it only works with real words and not with arbitrary strings.</p>
  
Tags: <machine-learning><natural-language-processing><classification><word2vec>
Title: How do I classify strings with possibly no meaning?
UserDisplayName: oezguensi
ThreadText: # How do I classify strings with possibly no meaning?
  
  (Asked by: username_0 on 2019-06-25)
  
  I am quite new to text classification.
  
  Using EAST text detection model, I get multiple strings that aren't words and most often have no meaning. For example, IDs, brand names, etc. I would like to classify them into two groups. Which models work the best and how should I preprocess the strings? I wanted to use Word2Vec, but I think it only works with real words and not with arbitrary strings.
  
  **Tags:** machine-learning, natural-language-processing, classification, word2vec
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-25. Score: 1
  
  I would just use a dictionary. A simple list lookup would tell you whether it's a recognised word or not. As an added bonus you can add some basic language processing, eg to identify inflected forms without listing them in your dictionary. Or use regular expressions to recognise ID numbers. ML is not really the right tool here.
  
  > **Comments:**
  > **username_0**: What do you mean with dictionary? the problem I have is I got strings such as „526//51“ which would be an ID I want to recognize. But also „A1213“ would be an ID. And unfortunately every ID just occurs once or twice. So I can‘t use stuff like tf-id  
  > **username_1**: A dictionary is basically a word list. For IDs you could use regular expression matching, like /[A-Z][0-9]+/ to recognise a letter followed by one or more digits. You could even do it the other way round: use a regular expression to recognise words (basically /[a-z]+/, possibly with a hyphen included), and treat everything else as a non-word/ID. The matching words you can then check against a dictionary/list, either of words or (in order to exclude them) brand names.  
  > **username_0**: I would like to use the power of labels and train a supervised model. What would be the best way to create embeddings. I read about different embedding methods such as hashing.  
  > **username_1**: It's the wrong tool.  
  
  ---
  
  