This was just a small debugging code snippet.
I needed the .mean(), since the output was not a single values and otherwise I would have to specify the gradients.
Anyway, even with nn.CrossEntropyLoss your function works:
x = torch.randn(10, 10)
target = torch.randint(0, 10, (10,))
lin1 = nn.Linear(10, 10)
lin2 = nn.Linear(10, 10)
criterion = nn.CrossEntropyLoss()

out = lin1(x)
out = RBF(out)
out = lin2(out)
loss = criterion(out, target)
loss.backward()
print(lin1.weight.grad)
print(lin2.weight.grad)

The error might be in another part of your code.
Make sure you are not using the .data attribute or use another library e.g. numpy in your calculations.
Also, you could post a (small and executable) code snippet, so that we can have a look.