thanks for the quick response @lewtun!
I was actually trying to pretrain roberta model on GCP’s TPU using HF’s Roberta Flax trining script. I’ve followed the following steps to do it - transformers/examples/flax/language-modeling at master · huggingface/transformers (github.com) 5
For the time being, I sidetracked the error by removing --push_to_hub flag when running the training script.