Hi!
So I have no idea what’s going on.
Here is my code.
PS.
The output is def a function of the input (model is a pretty good[93%] gender classifier).
def compute_saliency_maps(X, y, model):
    # Make sure the model is in "test" mode
    model.eval()
    
    # Wrap the input tensors in Variables
    X_var = Variable(X, requires_grad=True).cuda()
    y_var = Variable(y).cuda()
    
    scores = model(X_var)
    
    # Get the correct class computed scores.
    scores = scores.gather(1, y_var.view(-1, 1)).squeeze()
    
    # Backward pass, need to supply initial gradients of same tensor shape as scores.
    scores.backward(torch.FloatTensor([1.0,1.0,1.0,1.0]).cuda())
    
    # Get gradient for image.
    saliency = X_var.grad.data #Here X_var.grad is still None! What the hell?!