I would recommend to remove the usage of Variables, as they are deprecated since PyTorch 0.4, but your code should generally work.
Here is a small example showing that gradients are accumulated in the input using a simple model:
x = torch.randn(1, 10, requires_grad=True)
model = nn.Linear(10, 10)
output = model(x)

# Backward pass.
model.zero_grad()
output.mean().backward()
relevance_map = x.grad.cpu().numpy()[0]
print(relevance_map)