Thanks @ptrblck for the reply. I computed loss first and then called loss.backward as follows:
PyTorch
Loss function:
def cross_entropy_with_logits(logits, targets, batch_size):
    values = (targets * th.log(logits))
    reduced_values =  values.sum()
    result = - reduced_values/ batch_size

    return result

Computing loss
logits, activations = model.forward(X)
loss = cross_entropy_with_logits(logits, y, batch_size)

# backprop
loss.backward()

Printing gradients
for param in model.parameters():
     print(param.grad)