Hi @Tony-Y
Thank you for your code snippet.  It is very helpful.
I have one suggestion:
In order to make model.cuda() work, we need to wrap the self.centers with nn.Parameter() like:
self.centers = nn.Parameter(self.centers, requires_grad=False)

Also, if we are expecting the first dimension is the batch size, I update the
forward() function as follows:

    def forward(self, x):
        x = torch.unsqueeze(x, 1) - torch.unsqueeze(self.centers, 1)
        x = torch.exp(-0.5*(x/self.sigma)**2) / (self.sigma * np.sqrt(np.pi*2)) * self.delta
        x = x.sum(dim=-1)
        x = x/x.sum(dim=-1).unsqueeze(1) # normalization
        return x