Interesting in the tutorial we  don’t need to specify device = accelerator.device and does not need to push model to device model = model.to(device). Perhaps this is the problem in my script that only uses 1 TPU core?