There are two things:

The variable hidden is overwritten by output, hidden = model(data, hidden), so if you want the initial hidden state, you would have to instead do output, hidden_new = ....
The initial hidden state likely does not require grad, so you need hidden[0].requires_grad_() for the initial hidden state (for the final hidden state, new_hidden[0].retain_grad_() is good.

All this assumes you are indeed using the LSTM and hidden and new_hidden are tuples.
Best regards
Thomas