111179:

@staticmethod
    def forward(ctx, input):
        ctx.save_for_backward(input)
        out = torch.zeros_like(input).cuda()
        out[input > 0] = 1.0
        return out

    @staticmethod
    def backward(ctx, grad_output):
        input, = ctx.saved_tensors
        grad_input = grad_output.clone()
        grad = grad_input * 0.3 * F.threshold(1.0 - torch.abs(input), 0, 0)
        return grad



hey thanks for helping but i checked and it doesn’t seem to use your custom backward pass. it uses pytorches default i believe.
way to check- try using breakpoints maybe on spyder or jupyter. insert break points inside forward and backward.
2nd compare your loss values for both cuda and cpu.