<p>AlphaTensor by DeepMind is likely the most notable attempt to speed up matrix multiplication using AI.</p>
<p>As a note on the second question, likely the advancement here would be to attempt to analyze what the network is doing and build a rule set from it.  Running a neural network is faster than training it but nowhere near as fast as if we can translate it’s logic to rules.</p>

CC BY-SA 4.0
2024-08-14T13:56:34.183Z
So, basically the approach could work and could make it faster?
user172776
CC BY-SA 4.0
2024-08-14T15:58:55.007Z
DeepMind has shown evidence that this may be possible.  Paper is a year or two old and I'm not sure if any other researchers have explored the methods presented or the problem further.
foreverska
CC BY-SA 4.0
2024-08-14T12:39:50.253Z
2024-08-14T12:39:50.253Z
foreverska
<p>As I understand Neural Networks they have a slow training phase (quadratic or cubic time) and a fast (linear time) inference phase.
Also, the slow training phase comes from the requirement of doing matrix multiplications for which fast exact algorithms are not possible.</p>
<p><strong>Question</strong></p>
<ul>
<li>Were there any experiments before, where people tried to build a Neural Network to achieve approximate fixed size Matrix Multiplication?</li>
<li>Is it possible to train a Neural Network for calculating fixed size matrix multiplication and using this Neural Network Model (which then has fast inference) for training a second Neural Network (which then has fast inference and training)?</li>
</ul>

CC BY-SA 4.0
2024-08-13T22:30:58.453Z
matrix multiplication are not the bottleneck of NNs
Alberto
CC BY-SA 4.0
2024-08-14T04:08:35.803Z
That's the first time I hear this... "The bottleneck for deep neural networks is matrix multiply. [...] As a result, any good deep learning system must
involve efficient matrix multiplication." https://www.cs.cornell.edu/courses/cs4787/2019sp/notes/lecture11.pdf
user172776
CC BY-SA 4.0
2024-08-14T18:04:27.503Z
one thing is theory, another one is practice. matmul, and even more nn forward pass, are embarrassingly parallelizable, try fitting a GP with matrix inversion, or a KNN calculating the distances, and you’ll realize what i mean :)
Alberto
CC BY-SA 4.0
2024-08-15T04:43:49.073Z
I don't think that a few SIMD operations will make a more-than-quadratic runtime complexity become linear runtime. If MatMul is not the bottleneck for NNs, what is it in your opinion?
user172776
<neural-networks><training><efficiency>
Matrix Multiplication using a Neural Network
user172776
# Matrix Multiplication using a Neural Network

(Asked by: username_0 on 2024-08-13)

As I understand Neural Networks they have a slow training phase (quadratic or cubic time) and a fast (linear time) inference phase.
Also, the slow training phase comes from the requirement of doing matrix multiplications for which fast exact algorithms are not possible.

**Question**

* Were there any experiments before, where people tried to build a Neural Network to achieve approximate fixed size Matrix Multiplication?
* Is it possible to train a Neural Network for calculating fixed size matrix multiplication and using this Neural Network Model (which then has fast inference) for training a second Neural Network (which then has fast inference and training)?

**Tags:** neural-networks, training, efficiency

> **Comments:**
> **username_1**: matrix multiplication are not the bottleneck of NNs  
> **username_0**: That's the first time I hear this... "The bottleneck for deep neural networks is matrix multiply. [...] As a result, any good deep learning system must involve efficient matrix multiplication." https://www.cs.cornell.edu/courses/cs4787/2019sp/notes/lecture11.pdf  
> **username_1**: one thing is theory, another one is practice. matmul, and even more nn forward pass, are embarrassingly parallelizable, try fitting a GP with matrix inversion, or a KNN calculating the distances, and you’ll realize what i mean :)  
> **username_0**: I don't think that a few SIMD operations will make a more-than-quadratic runtime complexity become linear runtime. If MatMul is not the bottleneck for NNs, what is it in your opinion?  

---

## 1 answers

### Answer by username_2 on 2024-08-14. Score: 1 | Accepted answer

AlphaTensor by DeepMind is likely the most notable attempt to speed up matrix multiplication using AI.

As a note on the second question, likely the advancement here would be to attempt to analyze what the network is doing and build a rule set from it. Running a neural network is faster than training it but nowhere near as fast as if we can translate it’s logic to rules.

> **Comments:**
> **username_0**: So, basically the approach could work and could make it faster?  
> **username_2**: DeepMind has shown evidence that this may be possible.  Paper is a year or two old and I'm not sure if any other researchers have explored the methods presented or the problem further.  

---

