There seems to be experimental support for sparse matrices in PyTorch. I’ve never used them before but maybe this will be helpful - torch.sparse 178
EDIT: You might want to have a look at this discussion 198 on GitHub regarding the state of sparse tensors in PyTorch.