I overrode the tokenizer loading code thus:
tokenizer = AutoTokenizer.from_pretrained("./EsperBERTo", max_len=512)

Then I ran it again with cli 512, config 514, it no longer crashes in the same way:
06/08/2021 09:23:56 - INFO - __main__ -   tokenizer: PreTrainedTokenizerFast(name_or_path='./EsperBERTo', vocab_size=52000, model_max_len=512, is_fast=True, padding_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'sep_token': '</s>', 'pad_token': '<pad>', 'cls_token': '<s>', 'mask_token': AddedToken("<mask>", rstrip=False, lstrip=True, single_word=False, normalized=False)})
06/08/2021 09:23:56 - INFO - __main__ -   Training new model from scratch
  0%|                                                                                                                                                                               | 0/975 [00:00<?, ?ba/s][WARNING|tokenization_utils_base.py:3170] 2021-06-08 09:23:59,091 >> Token indices sequence length is longer than the specified maximum sequence length for this model (610 > 512). Running this sequence through the model will result in indexing errors