Answers.Body: <p>If your main issue is dealing with new vocabulary, you could try using a parts-of-speech tagger as a pre-processing step. You would then effectively discover relationships between "noun" and "verb", which does not change with new words. Taggers usually can handle unknown words by using contextual information.</p>
  
  <p>So you'd tag the words with their word class labels, and use those for training.</p>
  
  <p>In 'application mode' you use p-o-s tags again, calculate the scores, and then map them back onto your original word tokens.</p>
  
  <p>This does of course loose you some information, as you're only dealing with N-V, rather than <em>bomb</em>-<em>plant</em> or <em>bomb</em>-<em>defuse</em>. To solve this I'd use a hybrid approach: for your known vocabulary you use the word tokens, whereas for unknown words you fall back on tags. If you train two classifiers, one with tokens, one with tags, you have the tags one as a 'safety net' to handle out-of-vocab words.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-18T10:30:53.647Z
Answers.LastActivityDate: 2018-05-18T10:30:53.647Z
Answers.UserDisplayName: Oliver Mason
Body: <p>Considering the scenario where supervised training data-set in the form of sentence will be given to train the machine</p>
  
  <blockquote>
    <p>The Bomb which had been planted by Terrorist on this morning was
    defused by the Counter Terrorist on joining hands with the
    Intelligence Force</p>
  </blockquote>
  
  <p>Input strings in the sentence containing each words are broken into tokenised
  arrays of single words with stop words removed.<br/> </p>
  
  <p>Each word in the given sentence gets assigned a label w1, w2 and so on i.e,<br/>
  <code>w2</code>  = Bomb<br/>
  <code>w6</code>  = planted<br/>
  <code>w13</code> = defused<br/></p>
  
  <p>Calculating the scores for individual word combinations, the result should yield something like:<br/>
  <code>w2.w6</code>  = Scores should be Positive (or > some threshold value)<br/>
  <code>w2.w13</code> = Scores should be Negative (or &lt; some threshold value)<br/></p>
  
  <p>In case of words with polarity changers<br/>
  Eg.: <em>Bomb wasn't/haven't/didn't got defused.</em><br/>
  The resulting scores should be positive<br/></p>
  
  <hr>
  
  <p>To accomplish this task I had implemented Sentiment Analysis with the threshold = 2.5 and ended up with the following scores<br/>
  <br/></p>
  
  <h2><strong>Actual Output:</strong></h2>
  
  <blockquote>
    <p>&lt; 2.5 : Low<br/>
    = 2.5 : Neutral<br/>
    > 2.5 : High<br/></p>
  </blockquote>
  
  <p><br/></p>
  
  <h2><strong>Expected Output:</strong></h2>
  
  <blockquote>
    <p><em>Case 1:</em> score = negative, since that bomb was defused or removed in the given sentence<br/>
    <em>Case 2:</em> score = positive, vice versa of "Case 1"<br/>
    <em>Case 3</em>: Otherwise score = 0, in case it can't predict either of the above two cases, it should be neutral <br/></p>
  </blockquote>
  
  <p>I am facing a severe problem every time I need to update the vocabulary list with upcoming new words that were not in the dictionary list, which is turning out to be Semi-supervised learning.</p>
  
  <p>Referring to the above sentence to calculate the <code>w(n-(1/2/3/...n)</code> and <code>wn</code> word with reference to word = <em>Bomb</em>. The final resulting score should yield as negative.</p>
  
  <p>So which machine learning algorithm would be appropriate that fits to yield a better solution and based on the given data set how will I train the machine to learn the above things?</p>
  
  <p>Finally should I try to implement by keeping the model persistence. So that it doesn’t have to be trained on each run.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-18T09:14:09.203Z
Comments.Text: It looks like you are trying to compute some kind of word relationships. Can you be a bit more explicit about the purpose of the scores, and what they should reflect?
Comments.UserDisplayName: Oliver Mason
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-18T09:55:09.817Z
Comments.Text: After scanning set of words in the given document file, the scores should reflect whether the bomb planted was defused or not at the end of the story @OliverMason
Comments.UserDisplayName: Nɪsʜᴀɴᴛʜ ॐ
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-18T12:20:37.663Z
Comments.Text: "whether the bomb planted was defused or not" Do you mean there is only one simple answer (defused/not defused)? why you don't try RNN with character level to overcome unknown words
Comments.UserDisplayName: Fadi Bakoura
Tags: <machine-learning><algorithm><learning-algorithms>
Title: Which machine learning algorithm is suitable for detecting text w.r.t set of words
UserDisplayName: Nɪsʜᴀɴᴛʜ ॐ
ThreadText: # Which machine learning algorithm is suitable for detecting text w.r.t set of words
  
  (Asked by: username_0 on 2018-05-17)
  
  Considering the scenario where supervised training data-set in the form of sentence will be given to train the machine
  
  > The Bomb which had been planted by Terrorist on this morning was
  > defused by the Counter Terrorist on joining hands with the
  > Intelligence Force
  
  Input strings in the sentence containing each words are broken into tokenised
  arrays of single words with stop words removed.
  
  Each word in the given sentence gets assigned a label w1, w2 and so on i.e,  
  `w2` = Bomb  
  `w6` = planted  
  `w13` = defused
  
  Calculating the scores for individual word combinations, the result should yield something like:  
  `w2.w6` = Scores should be Positive (or > some threshold value)  
  `w2.w13` = Scores should be Negative (or < some threshold value)
  
  In case of words with polarity changers  
  Eg.: *Bomb wasn't/haven't/didn't got defused.*  
  The resulting scores should be positive
  
  ---
  
  To accomplish this task I had implemented Sentiment Analysis with the threshold = 2.5 and ended up with the following scores
  
  ## **Actual Output:**
  
  > < 2.5 : Low  
  > = 2.5 : Neutral  
  > > 2.5 : High
  
  ## **Expected Output:**
  
  > *Case 1:* score = negative, since that bomb was defused or removed in the given sentence  
  > *Case 2:* score = positive, vice versa of "Case 1"  
  > *Case 3*: Otherwise score = 0, in case it can't predict either of the above two cases, it should be neutral
  
  I am facing a severe problem every time I need to update the vocabulary list with upcoming new words that were not in the dictionary list, which is turning out to be Semi-supervised learning.
  
  Referring to the above sentence to calculate the `w(n-(1/2/3/...n)` and `wn` word with reference to word = *Bomb*. The final resulting score should yield as negative.
  
  So which machine learning algorithm would be appropriate that fits to yield a better solution and based on the given data set how will I train the machine to learn the above things?
  
  Finally should I try to implement by keeping the model persistence. So that it doesn’t have to be trained on each run.
  
  **Tags:** machine-learning, algorithm, learning-algorithms
  > **Comments:**
  > **username_1**: It looks like you are trying to compute some kind of word relationships. Can you be a bit more explicit about the purpose of the scores, and what they should reflect?  
  > **username_0**: After scanning set of words in the given document file, the scores should reflect whether the bomb planted was defused or not at the end of the story @OliverMason  
  > **username_2**: "whether the bomb planted was defused or not" Do you mean there is only one simple answer (defused/not defused)? why you don't try RNN with character level to overcome unknown words  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-18. Score: 1
  
  If your main issue is dealing with new vocabulary, you could try using a parts-of-speech tagger as a pre-processing step. You would then effectively discover relationships between "noun" and "verb", which does not change with new words. Taggers usually can handle unknown words by using contextual information.
  
  So you'd tag the words with their word class labels, and use those for training.
  
  In 'application mode' you use p-o-s tags again, calculate the scores, and then map them back onto your original word tokens.
  
  This does of course loose you some information, as you're only dealing with N-V, rather than *bomb*-*plant* or *bomb*-*defuse*. To solve this I'd use a hybrid approach: for your known vocabulary you use the word tokens, whereas for unknown words you fall back on tags. If you train two classifiers, one with tokens, one with tags, you have the tags one as a 'safety net' to handle out-of-vocab words.
  
  
  ---
  
  