Based on the posted code snippets the first training and validation loops are identical to the “inference” code besides that the inference code uses the train_dataloader. It also seems you are not calling model.train() in your training loop. but based on your model definition this might not be necessary as the used layers do not change their behavior during training/validiation.
If my assumption is correct, I would try to reduce the use case a bit and check the predictions with a single sample to make sure the original training predictions are matching the inference code. My guess would be that the model loading might have failed in your inference code and you might be using an untrained model.