Not sure if there’s a memory leak – i.e., there’s insufficient data to determine this because your training loop runs only one time and then aborts due to high memory use.
When I reduce nfeatures = 15888 to nfeatures = 10000 I get
Allocated Memory: 871.82177734375 MB
New allocations: 2289.31884765625 MB
New allocations: 0.0 MB
New allocations: 0.0 MB
New allocations: 0.0 MB
New allocations: 0.0 MB
New allocations: 0.0 MB
...

The first 2Gb for the New allocations should be expected due to loss.backward()

but still accumulate 5GB of garbage after the first batch.

In your case, it should be also only a one time thing, but hard to tell because your loop aborts after one iteration. The reason why it’s not the case when you call forward only is that the .grad attributes are  set to None before your first backward call, I’d say.