Are you creating some synchronization points in your training loop, e.g. via print statements?
I assume I could profile the code via:
data = torch.randn(9298, 256)
target = torch.randint(0, 10, (9298,))
model = # your definition

If I remember correctly, you cannot install a newer PyTorch version, so I would at least want to make sure, we don’t have a regression in the latest version.