The performance difference seems to be related to numerical stability, particularly in F.log_softmax and nn.CrossEntropy (which I think relies on F.log_softmax). I believe this for the following reasons.

Actually, I had the opposite experience. I have had numerical stability issues before (more due to a custom implementation of a loss function) and thought that 1.0 might fix that because there was some bugfix related to F.log_softmax

https://github.com/pytorch/pytorch/pull/11866 16

However, I noticed zero difference between running the code in 0.4.1 and 1.0 though.
Potential explanation 1:
A potential explanation (assuming when you say “models significantly outperformed the float64 1.0.0 model” you mean predictive performance of the model?) is maybe that you previously used hyperparameters optimal for the previously “buggy” log_softmax, and these parameters are now not the best choice in 1.0 given that bug fix?
Potential explanation 2:
The bugfix mentioned above was not actually a bug fix and made things worse.