My code is as follows:
class MyReLU(torch.autograd.Function):

    @staticmethod
    def forward(ctx, input):
        ctx.save_for_backward(input)
        output = input.clamp(min=0)
        return output

    @staticmethod
    def backward(ctx, grad_output):
        input, = ctx.saved_variables
        grad_output[input < 0] = 0
        return grad_output

myRelu = MyReLU.apply

…
which runs well in the earlier version of pytorch, and fails in the latest version of pytorch
(up to yesterday), and gave out error message in the above first post. However, after just
happen to read a related post

  
    
    
    Custom autograd.Function: backward pass not called autograd
  
  
    Good afternoon! 
I’ve had this problem in my other thread already, but it isn’t really related, so I moved it to a new thread. 
Whenever I try using a custom autograd.Function (i.e., a function that has an explicit backward pass defined), and I combine it with any torch.nn module, the backward pass is never properly executed. 
Here’s an MWE containing a simple identity transform as an example: 

import torch
import torch.autograd
import torch.nn



# custom autograd function
class Identity(torch…
  


I modified the code to
class MyReLU(torch.autograd.Function):

    @staticmethod
    def forward(self, input):
        self.save_for_backward(input)
        output = input.clamp(min=0)
        return output

    @staticmethod
    def backward(self, grad_output):
        input, = self.saved_tensors
        grad_output[input < 0] = 0
        return grad_output

and it works now in the both old and newer version of pytorch! Interesting, but I don’t know why!
If you or someone can explain this, that would be appreciated!