Don’t think so - here memory is consumed to preserve original input tensor for backprop (mm op), so right matrix “sparsity” makes no difference. Maybe take a look at torch.utils.checkpoint if memory is an issue.