Solved by sgugger in post #3 
              
              
                Also note that now Accelerate automatically adds that dimension when doing a gather of 0d-tensors, so you can now just do: 
validation_losses.append(accelerator.gather(loss))