ResourceExhaustedError: Failed to allocate request for 64.00MiB (67108864B) on device ordinal 0
I'm training an XLM model from huggingface on a small dataset using Colab TPU. However, I get the following error:
<ResourceExhaustedError: Failed to allocate request for 64.00MiB (67108864B) on device ordinal 0>

I tried reducing the batch size and restarting the kernel and freeing memory but nothing is working.
Any help would be appreciated.
Can you give some additional info? like the number of layers and the batch size?<|||||>I'm following this notebook ->https://www.kaggle.com/dimasmunoz/text-classification-with-roberta-and-tpus
I've tried Roberta, BERT, DistilBert and all working okay on my dataset. Except for XLM !
`MODEL_NAME = 'xlm-mlm-en-2048'    ## 12 layers, 16 heads
MAX_LEN = 256
ARTIFACTS_PATH = '../artifacts/'

BATCH_SIZE = 4 * strategy.num_replicas_in_sync
EPOCHS = 3`<|||||>Worked fine for the EnghlishGerman model "xlm-mlm-ende-1024" with 6 layers, 8 heads