Hello everyone!
I have an input of this shape:
(num_samples, num_symbols, num_features, num_observations)

I would like to feed through this input through a neural network.
The neural network as (num_features, num_observations) shape input and (num_outputs) outputs, giving me (num_samples, num_symbols, num_outputs) when I apply along
def apply_along_axis(function, x, axis: int = 0):
    return torch.stack([
        function(x_i) for x_i in torch.unbind(x, dim=axis)
    ], dim=axis)

(from Pytorch equivalent of numpy.apply_along_axis 8 )
This is an operation that is made very often in my network, and I just discover that 42% of the time is spent in the apply_along_axis.
I wanted to know if there is better way to do this ? (the for loop seems to be quite slow)
Is there a possibility to create identical versions of the same “function” in the network and update all the weights automatically when grad is invoked ?
(I maybe have a similar question with more complex situation if there is good suggestion related to this issue)
Thanks!