This only happens when I am using matmul. If I replace matmul with a + or something (e.g.,
h_b = probs_b.view(B, 1) + self._weight_b

, as long as I use a batch size of 1)
there isn’t a memory leak.
The memory leak also happens when I wrap the forward pass in a no_grad and set the model to eval mode. It also happens if I uncomment the backwards pass and optimizer.step in the training code (so basically, it’s just doing a bunch of forward passes on different data).
Another weird thing is that gc isn’t reporting the tensors that are adding to memory_allocated. The tensors reported by gc (even the shared_tensors) are stable across batches, whereas the allocated memory is increasing.