Answers.Body: <p>I think you meant pattern recognition instead of pattern matching in you question. Because pattern matching has nothing to do with NNs as far as I know. So RNNs is the easiest architecture you could use for this task. Have a look at <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" rel="nofollow noreferrer">this post</a>. It's long, but it's very well written and explains why RNNs work well with sequential data.</p>
  <p>In brief, RNNs accumulate information about what was fed to them previously and store it in the hidden state vector. That information is a summary of a sequence which helps it to predict the output sequence. RNNs can be applied to variable length input and catch dependencies among different parts of the input (as opposed to dense layers that can be applied to a sequence only point-wise).
  Here's how exactly you can apply reccurent layers (<a href="https://www.tensorflow.org/guide/keras/rnn/" rel="nofollow noreferrer">2</a>, <a href="https://pytorch.org/docs/stable/generated/torch.nn.RNN.html" rel="nofollow noreferrer">3</a>).
  There are more advanced architectures like LSTMs. Also you can try transformers (<a href="http://jalammar.github.io/illustrated-transformer/" rel="nofollow noreferrer">4</a>) which use attention mechanism (<a href="https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html" rel="nofollow noreferrer">5</a>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-12T15:02:23.493Z
Answers.Text: solotsky Can you give an explanation on why you find RNN's to be good for finding abstract patterns? Also, My input sequence is just a word of about 40 characters so I doubt it would be very useful...
Answers.UserDisplayName: neel g
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-08T21:21:01.733Z
Answers.Text: Michael: NNs are used for pattern matching.
Answers.UserDisplayName: Brian O'Donnell
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-12T13:50:09.640Z
Answers.LastActivityDate: 2023-04-30T10:30:34.297Z
Answers.LastEditDate: 2023-04-30T10:30:34.297Z
Answers.UserDisplayName: Michał Słodki
Body: <p>I am trying to make a NN(probably with dense layers) to map a <strong>specific</strong> input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and predict the output of any other input I give it.</p>
  <p>I have 2 files - one with the inputs and another with all the corresponding outputs and would probably use a bunch of Dense Layers with word embeddings to vectorize it into higher dimensions. However, I cannot find any good resources out there for that.</p>
  <p>Does anyone know how to accomplish such an NN? Which architectures are best for pattern matching? examples, links, and other resources would be very welcome. I was considering using RNN's but found them not very good in the pattern matching tasks so had ditched them. I would still consider them if someone can provide a plausible explanation...</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-08T21:18:22.930Z
Comments.Text: Can you provide a sample of data?  The only clue you provided is word embeddings so I assume you have text.  Why do you speak of patterns?  Are you looking for specific phrases or sequences of words?  If so are you interested in semantics and not just syntax?
Comments.UserDisplayName: Brian O'Donnell
Tags: <machine-learning><deep-learning><prediction><pattern-recognition><architecture>
Title: How to use a NN for seq2seq tasks?
UserDisplayName: neel g
ThreadText: # How to use a NN for seq2seq tasks?
  
  (Asked by: username_0 on 2020-09-11)
  
  I am trying to make a NN(probably with dense layers) to map a **specific** input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and predict the output of any other input I give it.
  
  I have 2 files - one with the inputs and another with all the corresponding outputs and would probably use a bunch of Dense Layers with word embeddings to vectorize it into higher dimensions. However, I cannot find any good resources out there for that.
  
  Does anyone know how to accomplish such an NN? Which architectures are best for pattern matching? examples, links, and other resources would be very welcome. I was considering using RNN's but found them not very good in the pattern matching tasks so had ditched them. I would still consider them if someone can provide a plausible explanation...
  
  **Tags:** machine-learning, deep-learning, prediction, pattern-recognition, architecture
  
  > **Comments:**
  > **username_1**: Can you provide a sample of data?  The only clue you provided is word embeddings so I assume you have text.  Why do you speak of patterns?  Are you looking for specific phrases or sequences of words?  If so are you interested in semantics and not just syntax?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-09-12. Score: 0
  
  I think you meant pattern recognition instead of pattern matching in you question. Because pattern matching has nothing to do with NNs as far as I know. So RNNs is the easiest architecture you could use for this task. Have a look at [this post](http://karpathy.github.io/2015/05/21/rnn-effectiveness/). It's long, but it's very well written and explains why RNNs work well with sequential data.
  
  In brief, RNNs accumulate information about what was fed to them previously and store it in the hidden state vector. That information is a summary of a sequence which helps it to predict the output sequence. RNNs can be applied to variable length input and catch dependencies among different parts of the input (as opposed to dense layers that can be applied to a sequence only point-wise).
  Here's how exactly you can apply reccurent layers ([2](https://www.tensorflow.org/guide/keras/rnn/), [3](https://pytorch.org/docs/stable/generated/torch.nn.RNN.html)).
  There are more advanced architectures like LSTMs. Also you can try transformers ([4](http://jalammar.github.io/illustrated-transformer/)) which use attention mechanism ([5](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)).
  
  > **Comments:**
  > **username_0**: solotsky Can you give an explanation on why you find RNN's to be good for finding abstract patterns? Also, My input sequence is just a word of about 40 characters so I doubt it would be very useful...  
  > **username_1**: Michael: NNs are used for pattern matching.  
  
  ---
  
  