Body: <h3>Abstract</h3>
  
  <p>I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. This means a human can intervene in its categorization attempts and determine whether or not it was correct and have it adjust its weights accordingly (without having to retrain all over again).</p>
  
  <h3>Inputs</h3>
  
  <p>It is know that ALL input will follow these rules:</p>
  
  <ol>
  <li>Always of <span class="math-container">$N$</span> length</li>
  <li>All messages are transformed to eliminate unnecessary complexity</li>
  </ol>
  
  <p>Here's a brief overview of how an example message might be processed.</p>
  
  <p>Starting with a message <span class="math-container">$M$</span>:</p>
  
  <blockquote>
    <p>That's an interesting perspective. I think that you should consider adding more details to your point about the cat being too silly.</p>
  </blockquote>
  
  <p>The text is then transformed so that extra details are removed:</p>
  
  <blockquote>
    <p>thats an interesting perspective i think that you should consider adding more details to your point about the cat being too silly</p>
  </blockquote>
  
  <p>Then it's converted into a vector (appending <span class="math-container">$0$</span> to reach length <span class="math-container">$N$</span>) ready to be processed by the neural network:</p>
  
  <pre>[116, 104, 97, 116, 115, 32, 97, . . . , 0, 0, 0, 0]</pre>
  
  <h3>Ouputs</h3>
  
  <p>In my network, I wish all the outputs to be weighted on how well they fit in each category. I <strong>need</strong> multiple outputs. I'm not really focusing on one particular category per-say, but how well the message fits in all of them. </p>
  
  <p>Following the input <span class="math-container">$M$</span> I used as an example, I'd expect the outputs to look <em>something</em> like this after my vector has fed-forward:</p>
  
  <pre>Suggestive:  0.89042<br>Opinionated: 0.68703</pre>
  
  <p>The weight values for each output indicate the strength of the category in the overall message.</p>
  
  <p>From message <span class="math-container">$M$</span>:</p>
  
  <blockquote>
    <p>That's an interesting perspective.</p>
  </blockquote>
  
  <p>Would weigh the opinionated category as <span class="math-container">$0.68703$</span>.</p>
  
  <p>And:</p>
  
  <blockquote>
    <p>I think that you should consider adding more details to your point about the cat being too silly.</p>
  </blockquote>
  
  <p>Would weigh the suggestive category as <span class="math-container">$0.89042$</span>.</p>
  
  <h3>Summary and Questions</h3>
  
  <p>I'm interested in the architectural design choices of a network that would support my feature set. The main goal is to be able to train my network to categorize messages based on pre-trained (and live-trained) data. I'd like to know things like:</p>
  
  <ol>
  <li>What type of Neural Network I should use for this purpose? I've researched <a href="https://en.wikipedia.org/wiki/Long_short-term_memory" rel="nofollow noreferrer">LSTM</a> &amp; <a href="https://en.wikipedia.org/wiki/Recurrent_neural_network" rel="nofollow noreferrer">Recurrent</a> networks; which have been mentioned to be good at processing sequences (ie. messages).</li>
  <li>What considerations should I account for when creating this network?.</li>
  <li>How can the overall network support live-training so I can tell my network when its wrong and have it 'correct' itself without having to retrain completely? </li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-23T23:55:56.913Z
Comments.Text: How do you vectorize your inputs?
Comments.UserDisplayName: JahKnows
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-24T00:10:52.660Z
Comments.Text: @JahKnows I've considered converting each character into a float (ie. getting the code point of each) and also converting each word into a float (ie. having a word database with a float value assigned to each). Whichever method works best for my case will be how I vectorize inputs.
Comments.UserDisplayName: Anilla
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-24T07:41:35.950Z
Comments.Text: have you considered some techniques like bag-of-words, tf-idf and n-grams?
Comments.UserDisplayName: JahKnows
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-24T15:55:48.497Z
Comments.Text: (along with JK) my suggestion is try out LSA 1st to learn the concepts of converting words to vectors, and its similar to NN techniques but actually somewhat conceptually more basic using linear analysis. https://en.wikipedia.org/wiki/Latent_semantic_analysis also there are now several high quality open source off-the-shelf NN systems for NLP (natural language processing) that can be applied without learning all the theory.
Comments.UserDisplayName: vzn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-24T22:26:10.893Z
Comments.Text: @vzn Thanks. The goal for me is to learn everything from scratch. Is there any places you could point me towards that really break down NLP NN concepts?
Comments.UserDisplayName: Anilla
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-25T01:36:08.700Z
Comments.Text: just googling this ref by Kisler is comprehensive. there are many videos that may be esp helpful to beginners. _Introduction to Natural Language Processing (NLP)_  https://blog.algorithmia.com/introduction-natural-language-processing-nlp/
Comments.UserDisplayName: vzn
Tags: <neural-networks><machine-learning><recurrent-neural-networks>
Title: How can my Neural Network categorize message strings?
UserDisplayName: Anilla
ThreadText: # How can my Neural Network categorize message strings?
  
  (Asked by: username_0 on 2019-02-23)
  
  Abstract
  I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. This means a human can intervene in its categorization attempts and determine whether or not it was correct and have it adjust its weights accordingly (without having to retrain all over again).
  Inputs
  It is know that ALL input will follow these rules:
  
  Always of $N$ length
  All messages are transformed to eliminate unnecessary complexity
  
  Here's a brief overview of how an example message might be processed.
  Starting with a message $M$:
  
  That's an interesting perspective. I think that you should consider adding more details to your point about the cat being too silly.
  
  The text is then transformed so that extra details are removed:
  
  thats an interesting perspective i think that you should consider adding more details to your point about the cat being too silly
  
  Then it's converted into a vector (appending $0$ to reach length $N$) ready to be processed by the neural network:
  [116, 104, 97, 116, 115, 32, 97, . . . , 0, 0, 0, 0]
  Ouputs
  In my network, I wish all the outputs to be weighted on how well they fit in each category. I need multiple outputs. I'm not really focusing on one particular category per-say, but how well the message fits in all of them. 
  Following the input $M$ I used as an example, I'd expect the outputs to look something like this after my vector has fed-forward:
  Suggestive:  0.89042Opinionated: 0.68703
  The weight values for each output indicate the strength of the category in the overall message.
  From message $M$:
  
  That's an interesting perspective.
  
  Would weigh the opinionated category as $0.68703$.
  And:
  
  I think that you should consider adding more details to your point about the cat being too silly.
  
  Would weigh the suggestive category as $0.89042$.
  Summary and Questions
  I'm interested in the architectural design choices of a network that would support my feature set. The main goal is to be able to train my network to categorize messages based on pre-trained (and live-trained) data. I'd like to know things like:
  
  What type of Neural Network I should use for this purpose? I've researched LSTM & Recurrent networks; which have been mentioned to be good at processing sequences (ie. messages).
  What considerations should I account for when creating this network?.
  How can the overall network support live-training so I can tell my network when its wrong and have it 'correct' itself without having to retrain completely?
  
  **Tags:** neural-networks, machine-learning, recurrent-neural-networks
  > **Comments:**
  > **username_1**: How do you vectorize your inputs?  
  > **username_0**: @JahKnows I've considered converting each character into a float (ie. getting the code point of each) and also converting each word into a float (ie. having a word database with a float value assigned to each). Whichever method works best for my case will be how I vectorize inputs.  
  > **username_1**: have you considered some techniques like bag-of-words, tf-idf and n-grams?  
  > **username_2**: (along with JK) my suggestion is try out LSA 1st to learn the concepts of converting words to vectors, and its similar to NN techniques but actually somewhat conceptually more basic using linear analysis. https://en.wikipedia.org/wiki/Latent_semantic_analysis also there are now several high quality open source off-the-shelf NN systems for NLP (natural language processing) that can be applied without learning all the theory.  
  > **username_0**: @vzn Thanks. The goal for me is to learn everything from scratch. Is there any places you could point me towards that really break down NLP NN concepts?  
  > **username_2**: just googling this ref by Kisler is comprehensive. there are many videos that may be esp helpful to beginners. _Introduction to Natural Language Processing (NLP)_  https://blog.algorithmia.com/introduction-natural-language-processing-nlp/  
  
  ---
  
  ## 0 answers
  
  