After following up on your suggestion to investigate the training loop, I think I narrowed down the issue to a mal-alignment between X and Y in my dataloader. I’m still not 100% sure of the root cause, but it seems to be due to an error in my mental model of when pandas may reorder data. When I manually join X and Y before converting them to tensors, I can overfit as expected (same appearance as the above plot) but reassuringly I can also reproduce the overfit values outside of the training loop.