I found the issue. I was storing the loss from the function call in a tensor and this was never getting freed, for some reason. More interestingly, memory was consumed more quickly for cross entropy, perhaps because it consists of two operations instead of one, hence why NLL appeared to have no memory leak when it did.
What are the rules by which a tensor is garbage collected? Why does overwriting a tensor not free it?