Hello all,
I have been trying to recreate the Softmargin Softmax function found here: Softmargin Softmax paper.
I took the current Softmax function in Activation.py and changed the the forward step with my custom Python calculation.
def forward(self, input):
        m = 0
        input = input.exp()
        sumexp = torch.sum(input,dim=1)
        expm = math.exp(-m)
    
        # calculate softmargin softmax
        for x in range (input.size(0)):
            for y in range(input.size(1)):
                input[x,y] = (input[x,y]*expm)/(sumexp[x] - input[x,y] + (input[x,y] * expm))
         
        #normalize the weights
        sumnorm = torch.sum(input,dim=1)
        for x in range (input.size(0)):
            for y in range(input.size(1)):
                input[x,y] = input[x,y]/sumnorm[x]

        return input

If I now run my model, it does not crash, but seems to keep calculating something whitout progressing in the model.
Did I look over some important step? Or is it just so inefficient?
Any help is appreciated.
edit
The function appears to be running very,very slow.
Does somebody know how to speed it up?