Answers.Body: <p>If you're referring to the <em>Meta-Learning with Memory-Augmented Neural Networks</em> <a href="http://proceedings.mlr.press/v48/santoro16.html" rel="nofollow noreferrer">paper</a>, the premise is that
  In some cases, we wish to perform meta-learning when we are only provided with small (or no) data. For example - in one/few-shot learning we are expected to provide a label with only a few tagged samples. To do so, we use two techniques</p>
  <ul>
  <li>External memory module - we initialize a container of knowledge that is called upon to respond to challenging circumstances</li>
  <li>Label shuffling - labels are presented one time-step after their corresponding sample, which helps with dealing with simply learning the mapping <span class="math-container">$sample\rightarrow label$</span> without generalizing. More specifically, we feed the network with <span class="math-container">$(x_1, null),(x_2,y_1),...,(x_{t+1},y_t)$</span></li>
  </ul>
  <p>In addition, samples are shuffled across different datasets (samples from different datasets may appear in the same sequence). This encourages the network to use the memory module and extract the relevant label once the corresponding sample is provided.</p>
  <p>In terms of actual implementation, memory retrieval is based on a general similarity between vectors (I suggest going over the paper for more details).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-28T13:25:22.690Z
Answers.LastActivityDate: 2023-01-28T13:25:22.690Z
Answers.UserDisplayName: Hadar Sharvit
Body: <p>How does the memory augmented neural network (MANN) work? How can I make a simple MANN with a vanilla neural network especially without a recurrent network?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-23T14:38:59.247Z
Comments.Text: Are you referring to a specific paper/implementation?
Comments.UserDisplayName: Rexcirus
Tags: <neural-networks><deep-learning><tensorflow><pytorch><memory>
Title: How does the memory augmented neural network work, and how to make a simple implementation?
UserDisplayName: Eka
ThreadText: # How does the memory augmented neural network work, and how to make a simple implementation?
  
  (Asked by: username_0 on 2023-01-28)
  
  How does the memory augmented neural network (MANN) work? How can I make a simple MANN with a vanilla neural network especially without a recurrent network?
  
  **Tags:** neural-networks, deep-learning, tensorflow, pytorch, memory
  
  > **Comments:**
  > **username_1**: Are you referring to a specific paper/implementation?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-28. Score: 1
  
  If you're referring to the *Meta-Learning with Memory-Augmented Neural Networks* [paper](http://proceedings.mlr.press/v48/santoro16.html), the premise is that
  In some cases, we wish to perform meta-learning when we are only provided with small (or no) data. For example - in one/few-shot learning we are expected to provide a label with only a few tagged samples. To do so, we use two techniques
  
  * External memory module - we initialize a container of knowledge that is called upon to respond to challenging circumstances
  * Label shuffling - labels are presented one time-step after their corresponding sample, which helps with dealing with simply learning the mapping $sample\rightarrow label$ without generalizing. More specifically, we feed the network with $(x\_1, null),(x\_2,y\_1),...,(x\_{t+1},y\_t)$
  
  In addition, samples are shuffled across different datasets (samples from different datasets may appear in the same sequence). This encourages the network to use the memory module and extract the relevant label once the corresponding sample is provided.
  
  In terms of actual implementation, memory retrieval is based on a general similarity between vectors (I suggest going over the paper for more details).
  
  
  ---
  
  