who knows the most evil autograd tricks competition


Note as well that tom’s code will create undestructible ref cycle and will leak memory on top of being very wrong 
The original purpose of save_for_backward() is actually to avoid this cycle 
I would suggest this NotSoDangerousDot:
class NotSoDangerousDot(torch.autograd.Function):
    @staticmethod
    def forward(ctx, a, b):
        prod = a * b
        ctx.save_for_backward(b)
        ctx.prod = prod
        return prod.sum()

    @staticmethod
    def backward(ctx, grad_out):
        b, = ctx.saved_tensors
        prod = ctx.prod
        return grad_out * b, grad_out * (prod / b)

Which might be slightly slower but should be correct in all ways! (cheating the autograd version checker is bad! haha)