Answers.Body: <p>You already figured out much of the problem. You can solve it with sequence models like LSTM/GRU.</p>
  <p>One-hot encode word-types. Assume there are types of [properNoun, adjective, noun] as you said. Then &quot;Mike&quot; will be represented as a vector, <code>[1,0,0]</code>, &quot;fast&quot; as <code>[0,1,0]</code>, and &quot;airplane&quot; as <code>[0,0,1]</code>.</p>
  <p>Summing up these, you will train a model that takes <code>[[1,0,0], [0,1,0], [0,0,1]]</code> as input and return binary classification result.
  Thus you can use LSTM/GRU with <code>return_sequences = False</code>.</p>
  <p>An example with Keras is <a href="https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/" rel="nofollow noreferrer">here</a>.</p>
  <p>A few things you should be careful about:</p>
  <ul>
  <li>I have shown 2D input here but sequence models take 3D input in form of (batch_size, time_steps, number_of_features) so you will need to reshape even if you train with batch size 1. More details are <a href="https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/" rel="nofollow noreferrer">here</a>.</li>
  <li>Prepad the time dimension with 0 vector, so that number of time steps will be equal for all samples. More details are <a href="https://machinelearningmastery.com/data-preparation-variable-length-input-sequences-sequence-prediction/" rel="nofollow noreferrer">here</a>.</li>
  </ul>
  <p>For multi-word type, you can create multiple data instances and train the network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-25T16:30:06.377Z
Answers.LastActivityDate: 2021-07-25T16:30:06.377Z
Answers.UserDisplayName: meliksahturker
Body: <p>I would like to train an LSTM neural network to either &quot;approve&quot; or &quot;reject&quot; a string based on the word-type sequence.
  For instance: &quot;Mike's Airplane&quot; would output &quot;approved&quot;, but &quot;Airplane Mike's&quot; would output &quot;reject&quot;.
  My method for doing this is to decompose the string into an array of words.
  eg.</p>
  <pre><code>[&quot;Mike's&quot;, &quot;Airplane&quot;]
  </code></pre>
  <p>, then convert the array of words to an array of <strong>word-types</strong> since the actual word is irrelevant.
  The word types (pronoun, noun, adjective etc.) are defined constants having numerical values.
  eg.</p>
  <pre><code>const wordtypes={propernoun:1, adjective:2, noun:3, ownername:4};
  console.log(wordtypes.propernoun); // 1
  </code></pre>
  <p>Mike's Fast Airplane is</p>
  <pre><code>[&quot;Mike's&quot;, &quot;Fast&quot;, &quot;Airplane&quot;] 
  </code></pre>
  <p>which becomes:</p>
  <pre><code>input:[properNoun, adjective, noun]
  output: &quot;approve&quot;
  </code></pre>
  <p><strong>properNoun</strong> represents the first word(<em>Mike's</em>),
  <strong>adjective</strong> the second word(<em>Fast</em>),
  and <strong>noun</strong> the third word(<em>Airplane</em>).</p>
  <p>I would then like to use this array to train a Neural Network so that it can approve/reject other word-type sequences.</p>
  <p>I am concerned with the methodology/algorithm rather than the syntax; I'm extremely new to Machine Learning and Artificial Neural Networks, so, I am using brain.js and NodeJS because they're relatively easy to use.</p>
  <ul>
  <li><p>I would like to input multiple parameters for a single word because
  many words have multiple word types (depending on the context). For
  example, a word can be both a &quot;noun&quot; and a &quot;verb&quot;. How do I represent this input?</p>
  </li>
  <li><p>Is this a good application for LSTM? Or is there a better-suited ML<br />
  algorithm? My dilemma is in deriving the proper inputs &amp; methodology
  to effectively train the Neural Network.</p>
  </li>
  <li><p>How is my methodology for accomplishing this approval system?</p>
  </li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-22T21:09:43.110Z
Comments.Text: Language is complicated. Me and my friend know two Mikes, one of whom is a pilot, and we call him "Airplane Mike". I ask "Whose drink is this?" and my friend replies "Airplane Mike's". Contrived? Yes. This kind of construct is a real problem for NLP systems? Yes. It doesn't stop you making up any set of word rules you like and training an LSTM to learn it, but it may help if you explain whether your goal is just an abstract rule learning thing for teaching yourself RNNs, or whether you intend to use it for real world language processing, or for some other purpose.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-23T01:26:54.747Z
Comments.Text: Yes, it's a real-world application, but not NLP. It's actually **input validation** for a web-based system. 
  For example:
  input: [ adj, noun] output: "reject"
  input: [ noun, noun, noun] output: "reject"
  input: [ pronoun, adj, noun] output: "approve"
  input: [pronoun, noun] output: "approve"
  etc.
  The input is of variable length; "noun", "pronoun" etc are defined as integer constants.
Comments.UserDisplayName: Michael Lyons
Tags: <neural-networks><long-short-term-memory><input-layer>
Title: Using a Neural Network (LSTM) to approve/reject word-type sequences
UserDisplayName: Michael Lyons
ThreadText: # Using a Neural Network (LSTM) to approve/reject word-type sequences
  
  (Asked by: username_0 on 2021-07-22)
  
  I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence.
  For instance: "Mike's Airplane" would output "approved", but "Airplane Mike's" would output "reject".
  My method for doing this is to decompose the string into an array of words.
  eg.
  
  ```
  ["Mike's", "Airplane"]
  
  ```
  
  , then convert the array of words to an array of **word-types** since the actual word is irrelevant.
  The word types (pronoun, noun, adjective etc.) are defined constants having numerical values.
  eg.
  
  ```
  const wordtypes={propernoun:1, adjective:2, noun:3, ownername:4};
  console.log(wordtypes.propernoun); // 1
  
  ```
  
  Mike's Fast Airplane is
  
  ```
  ["Mike's", "Fast", "Airplane"] 
  
  ```
  
  which becomes:
  
  ```
  input:[properNoun, adjective, noun]
  output: "approve"
  
  ```
  
  **properNoun** represents the first word(*Mike's*),
  **adjective** the second word(*Fast*),
  and **noun** the third word(*Airplane*).
  
  I would then like to use this array to train a Neural Network so that it can approve/reject other word-type sequences.
  
  I am concerned with the methodology/algorithm rather than the syntax; I'm extremely new to Machine Learning and Artificial Neural Networks, so, I am using brain.js and NodeJS because they're relatively easy to use.
  
  * I would like to input multiple parameters for a single word because
    many words have multiple word types (depending on the context). For
    example, a word can be both a "noun" and a "verb". How do I represent this input?
  * Is this a good application for LSTM? Or is there a better-suited ML  
    algorithm? My dilemma is in deriving the proper inputs & methodology
    to effectively train the Neural Network.
  * How is my methodology for accomplishing this approval system?
  
  **Tags:** neural-networks, long-short-term-memory, input-layer
  
  > **Comments:**
  > **username_1**: Language is complicated. Me and my friend know two Mikes, one of whom is a pilot, and we call him "Airplane Mike". I ask "Whose drink is this?" and my friend replies "Airplane Mike's". Contrived? Yes. This kind of construct is a real problem for NLP systems? Yes. It doesn't stop you making up any set of word rules you like and training an LSTM to learn it, but it may help if you explain whether your goal is just an abstract rule learning thing for teaching yourself RNNs, or whether you intend to use it for real world language processing, or for some other purpose.  
  > **username_0**: Yes, it's a real-world application, but not NLP. It's actually **input validation** for a web-based system.  For example: input: [ adj, noun] output: "reject" input: [ noun, noun, noun] output: "reject" input: [ pronoun, adj, noun] output: "approve" input: [pronoun, noun] output: "approve" etc. The input is of variable length; "noun", "pronoun" etc are defined as integer constants.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-07-25. Score: 0 | Accepted answer
  
  You already figured out much of the problem. You can solve it with sequence models like LSTM/GRU.
  
  One-hot encode word-types. Assume there are types of [properNoun, adjective, noun] as you said. Then "Mike" will be represented as a vector, `[1,0,0]`, "fast" as `[0,1,0]`, and "airplane" as `[0,0,1]`.
  
  Summing up these, you will train a model that takes `[[1,0,0], [0,1,0], [0,0,1]]` as input and return binary classification result.
  Thus you can use LSTM/GRU with `return_sequences = False`.
  
  An example with Keras is [here](https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/).
  
  A few things you should be careful about:
  
  * I have shown 2D input here but sequence models take 3D input in form of (batch\_size, time\_steps, number\_of\_features) so you will need to reshape even if you train with batch size 1. More details are [here](https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/).
  * Prepad the time dimension with 0 vector, so that number of time steps will be equal for all samples. More details are [here](https://machinelearningmastery.com/data-preparation-variable-length-input-sequences-sequence-prediction/).
  
  For multi-word type, you can create multiple data instances and train the network.
  
  
  ---
  
  