Hi Thomas,
in the meantime Natalia opened a RFC issue: RFC: Should matmuls use tf32 by default? · Issue #67384 · pytorch/pytorch · GitHub 9 .
I want to add: Thanks for bringing it up! It’s important that users write about issues they see for the PyTorch developers to get a better idea of how intuitive default behaviour like this is and where to strike the balance between “maximizing perfomance” and “avoiding surprises”.
Best regards
Thomas