Answers.Body: <p>AlphaTensor by DeepMind is likely the most notable attempt to speed up matrix multiplication using AI.</p>
  <p>As a note on the second question, likely the advancement here would be to attempt to analyze what the network is doing and build a rule set from it.  Running a neural network is faster than training it but nowhere near as fast as if we can translate it’s logic to rules.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-14T13:56:34.183Z
Answers.Text: So, basically the approach could work and could make it faster?
Answers.UserDisplayName: user172776
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-14T15:58:55.007Z
Answers.Text: DeepMind has shown evidence that this may be possible.  Paper is a year or two old and I'm not sure if any other researchers have explored the methods presented or the problem further.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-14T12:39:50.253Z
Answers.LastActivityDate: 2024-08-14T12:39:50.253Z
Answers.UserDisplayName: foreverska
Body: <p>As I understand Neural Networks they have a slow training phase (quadratic or cubic time) and a fast (linear time) inference phase.
  Also, the slow training phase comes from the requirement of doing matrix multiplications for which fast exact algorithms are not possible.</p>
  <p><strong>Question</strong></p>
  <ul>
  <li>Were there any experiments before, where people tried to build a Neural Network to achieve approximate fixed size Matrix Multiplication?</li>
  <li>Is it possible to train a Neural Network for calculating fixed size matrix multiplication and using this Neural Network Model (which then has fast inference) for training a second Neural Network (which then has fast inference and training)?</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-13T22:30:58.453Z
Comments.Text: matrix multiplication are not the bottleneck of NNs
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-14T04:08:35.803Z
Comments.Text: That's the first time I hear this... "The bottleneck for deep neural networks is matrix multiply. [...] As a result, any good deep learning system must
  involve efficient matrix multiplication." https://www.cs.cornell.edu/courses/cs4787/2019sp/notes/lecture11.pdf
Comments.UserDisplayName: user172776
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-14T18:04:27.503Z
Comments.Text: one thing is theory, another one is practice. matmul, and even more nn forward pass, are embarrassingly parallelizable, try fitting a GP with matrix inversion, or a KNN calculating the distances, and you’ll realize what i mean :)
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-15T04:43:49.073Z
Comments.Text: I don't think that a few SIMD operations will make a more-than-quadratic runtime complexity become linear runtime. If MatMul is not the bottleneck for NNs, what is it in your opinion?
Comments.UserDisplayName: user172776
Tags: <neural-networks><training><efficiency>
Title: Matrix Multiplication using a Neural Network
UserDisplayName: user172776
ThreadText: # Matrix Multiplication using a Neural Network
  
  (Asked by: username_0 on 2024-08-13)
  
  As I understand Neural Networks they have a slow training phase (quadratic or cubic time) and a fast (linear time) inference phase.
  Also, the slow training phase comes from the requirement of doing matrix multiplications for which fast exact algorithms are not possible.
  
  **Question**
  
  * Were there any experiments before, where people tried to build a Neural Network to achieve approximate fixed size Matrix Multiplication?
  * Is it possible to train a Neural Network for calculating fixed size matrix multiplication and using this Neural Network Model (which then has fast inference) for training a second Neural Network (which then has fast inference and training)?
  
  **Tags:** neural-networks, training, efficiency
  
  > **Comments:**
  > **username_1**: matrix multiplication are not the bottleneck of NNs  
  > **username_0**: That's the first time I hear this... "The bottleneck for deep neural networks is matrix multiply. [...] As a result, any good deep learning system must involve efficient matrix multiplication." https://www.cs.cornell.edu/courses/cs4787/2019sp/notes/lecture11.pdf  
  > **username_1**: one thing is theory, another one is practice. matmul, and even more nn forward pass, are embarrassingly parallelizable, try fitting a GP with matrix inversion, or a KNN calculating the distances, and you’ll realize what i mean :)  
  > **username_0**: I don't think that a few SIMD operations will make a more-than-quadratic runtime complexity become linear runtime. If MatMul is not the bottleneck for NNs, what is it in your opinion?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-08-14. Score: 1 | Accepted answer
  
  AlphaTensor by DeepMind is likely the most notable attempt to speed up matrix multiplication using AI.
  
  As a note on the second question, likely the advancement here would be to attempt to analyze what the network is doing and build a rule set from it. Running a neural network is faster than training it but nowhere near as fast as if we can translate it’s logic to rules.
  
  > **Comments:**
  > **username_0**: So, basically the approach could work and could make it faster?  
  > **username_2**: DeepMind has shown evidence that this may be possible.  Paper is a year or two old and I'm not sure if any other researchers have explored the methods presented or the problem further.  
  
  ---
  
  