Body: <p>The forward pass in neural network can be written as g(Wx+b). W is the weights Matrix, x is the input vector and b is bias, and g the non linearity, the activation function. However x can have more than one dimension if the input is a image it becomes 2D. Also if we use minibatches we stack images and becomes 3D.</p>
  <p>I think that while using images in 2D for x would require W to becomes 3D, I think that stacking images for batches do not require W to become 4D since W would be the same for all the slices. A useless memory  usage increase.</p>
  <p>Is there any general purpose algorithm that scales to N dimensions? So I can do M minibatches with N dimensions in one go?</p>
  <p>Looking for the algorithm for instructional purposes. Something programming-like, I already tried abstract tensor stuff and I'm not even sure it is what I need</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-31T10:47:29.390Z
Comments.Text: AxBxC matmul CxZ = AxBxZ… so having additional dimensions don’t really matter
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-31T21:11:57.307Z
Comments.Text: Can you show an algorithm for that? I cannot find it
Comments.UserDisplayName: CoffeDeveloper
Tags: <algorithm-request><tensor><forward-pass>
Title: Implement tensor operation with mini batches instead of Matrix multiplication in forward pass
UserDisplayName: CoffeDeveloper
ThreadText: # Implement tensor operation with mini batches instead of Matrix multiplication in forward pass
  
  (Asked by: username_0 on 2024-03-30)
  
  The forward pass in neural network can be written as g(Wx+b). W is the weights Matrix, x is the input vector and b is bias, and g the non linearity, the activation function. However x can have more than one dimension if the input is a image it becomes 2D. Also if we use minibatches we stack images and becomes 3D.
  
  I think that while using images in 2D for x would require W to becomes 3D, I think that stacking images for batches do not require W to become 4D since W would be the same for all the slices. A useless memory usage increase.
  
  Is there any general purpose algorithm that scales to N dimensions? So I can do M minibatches with N dimensions in one go?
  
  Looking for the algorithm for instructional purposes. Something programming-like, I already tried abstract tensor stuff and I'm not even sure it is what I need
  
  **Tags:** algorithm-request, tensor, forward-pass
  > **Comments:**
  > **username_1**: AxBxC matmul CxZ = AxBxZ… so having additional dimensions don’t really matter  
  > **username_0**: Can you show an algorithm for that? I cannot find it  
  
  ---
  
  ## 0 answers
  
  