Coming back to the original question: I hacked around it for now by not using the self.save_for_backward mechanism, but instead creating a “self.sparse_weights” and setting it to None in the constructor, and doing this in forward:
    def forward(self, sparse_weights, x):
        if self.sparse_weights is None:
            self.sparse_weights = sparse_weights
        return torch.mm(self.sparse_weights, x)

Of course this is a terrible hack, and it completely rules out doing some backprop on that fixed matrix at some point. But at least it doesn’t leak memory.
If you’re wondering why I didn’t pass the fixed matrix to the constructor in the first place: that was my original solution, but then I got stuck keeping the CPU/GPU versions flexible. I think this solution is better at least in that respect…
But for one of the experts scanning the thread: This issue isn’t yet solved. The save_for_backward doesn’t work for me.