This is peculiar, but thanks for testing this thoroughly. I’ve had a quick look at PyTorch Adam 42 (seems fine) vs. TensorFlow Adam 36 (complex, but also seems fine) vs. Keras Adam 16 (also seems fine), and can’t spot any issues, but perhaps someone more observant will.