Update: I tried by adding model.save_pretrained(directory_name) within the function, such that you can access it after training, but this gives me an error:
Configuration saved in /content/config.json
---------------------------------------------------------------------------
ProcessExitedException                    Traceback (most recent call last)
<ipython-input-28-a91f3c0bb4fd> in <module>()
      1 from accelerate import notebook_launcher
      2 
----> 3 notebook_launcher(training_function)

3 frames
/usr/local/lib/python3.7/dist-packages/torch/multiprocessing/spawn.py in join(self, timeout)
    134                     error_pid=failed_process.pid,
    135                     exit_code=exitcode,
--> 136                     signal_name=name
    137                 )
    138             else:

ProcessExitedException: process 7 terminated with signal SIGKILL

This is probably because the model is still replicated on 8 TPU cores?