So the bummer is, autograd is not supported for sparse matrices. Sorry for not looking closely when i replied last.
We dont have any tests in test_autograd for sparse either.
We used sparse directly in nn layers such as nn.Embedding at the implementation level.