@ptrblck
How to make these weights learnable.
Am I doing it right here
class Model(nn.Module):
        def __init__(self, batch_size):
           super(Model, self).__init__()
           self.attention1   = nn.Parameter(torch.Tensor(self.batch_size,16, 32))
           self.attention2   = nn.Parameter(torch.Tensor(self.batch_size,1, 16))  

Attention Mechanism in Forward. The input here is the output after few CNN operations with Shape
[BatchSize X DimFeature X Length]   = [128 X 32 X 64]

       """ Get Attention Weights """
        attn = input    
        attention = attn.permute(0, 2, 1).matmul(self.attention1)
        attention = attention.matmul(self.attention2)
        attention = self.relu(attention)      
        attention = attention.view(attention.size(0), -1)
        attention = F.softmax(attention, 1)  
        """ Multiply Attention Weights with Audio Frames"""
        input = input * attention.unsqueeze(1)          #To Make it comptable with BatchSize X FeatureDim X FixedLength


@ptrblck Is this okay now?
I have also another question when I use nn.Softmax in place of F.softmax(attention, 1), why doesn’t it work ?