Hi,
Before trying to reinvent the wheel for my model I would like to check if something like the following
is already implemented.
Does the current implementation tries to perform the most optimal set of tensor product
based on the size of parent/children tensors?
A simple example is a product between the following matrices A * B* C = (100 x 100) * (100 x 10) * (10 x 1).
If I’m correct this should be a relatively common operation, especially during the backward-pass
due to the chain rule.
Performing (A*B) * C should result in 200900 flops, while A * (B * C) only 21800.