Hi
I am running a small model with small batch size on TPU with 8 cores, and when it calls
 xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
It always goes to out of memory on google cloud, and I am not sure where is the issue, I tried to reduce the batch size as much as possible still the error is there. thanks