Hey @skoaster10s
Another random component to consider is dropout. If you run the model in train mode, your dropout will still be active. For inference, you should try setting your model to model.eval()
Let everyone know if that was the issue!