Answers.Body: <p>Position-Wise basically means that the architecture takes position of a given value in the vector into considiration, and changing position of a given value in the input will impact the output.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-29T15:54:35.517Z
Answers.LastActivityDate: 2024-04-29T15:54:35.517Z
Answers.UserDisplayName: Saad El Kouari
Body: <p>I understand the architecture of a position-wise feed-forward network as described in (<a href="https://nn.labml.ai/transformers/feed_forward.html" rel="nofollow noreferrer">https://nn.labml.ai/transformers/feed_forward.html</a>). However I do not understand what &quot;position wise&quot; refers to. I assume &quot;position&quot; refers to where a data point is inside the input vector/matrix. What makes this architecture &quot;position-wise&quot;?</p>
  
Tags: <transformer><feedforward-neural-networks>
Title: What does "position-wise" fully connected mean?
UserDisplayName: Homer Sanchez
ThreadText: # What does "position-wise" fully connected mean?
  
  (Asked by: username_0 on 2024-04-29)
  
  I understand the architecture of a position-wise feed-forward network as described in (<https://nn.labml.ai/transformers/feed_forward.html>). However I do not understand what "position wise" refers to. I assume "position" refers to where a data point is inside the input vector/matrix. What makes this architecture "position-wise"?
  
  **Tags:** transformer, feedforward-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-29. Score: 0 | Accepted answer
  
  Position-Wise basically means that the architecture takes position of a given value in the vector into considiration, and changing position of a given value in the input will impact the output.
  
  
  ---
  
  