Tried to re-run, cleaned the .cache folder, all to the same end. The error remains. When I start the script the CPU peaks for a brief moment, until the error occurs. Let me try to provide you some more context:
txt_files = [str(i) for i in PATH_DATA.glob('all_*')]

datasets = load_dataset(
    path='text',
    data_files=txt_files[0],
    split=None,
)

#datasets['train'] = datasets['train'].select(range(1000))
datasets['train'], datasets['test'] = datasets['train'].train_test_split(test_size=0.02, shuffle=True, seed=1234).values()
datasets['train'], datasets['val']  = datasets['train'].train_test_split(test_size=0.02, shuffle=True, seed=1234).values()

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True)

datasets = datasets.map(
    lambda sequence: tokenizer(sequence['text'], return_special_tokens_mask=True),
    batched=True,
    batch_size=1000,
    num_proc=psutil.cpu_count(),
    remove_columns=['text'],
)

Output of the dataset after splitting:
DatasetDict({
    train: Dataset({
        features: ['text'],
        num_rows: 210759
    })
    test: Dataset({
        features: ['text'],
        num_rows: 4390
    })
    val: Dataset({
        features: ['text'],
        num_rows: 4302
    })
})

Note that I only load text files from txt_files . MODEL_NAME equals 'roberta-base'. I am note quite sure what you refer to in terms of dill.dumps tho. Sorry, I am still relatively new to the ecosystem …
EDIT1: As indicated, if I set num_proc=1 the code runs fluently.
EDIT2: By the way, I run into the same issue if I use other lambda functions that involve anything that is external to the mapping procedure. For example, I tried to debug it by simply returning a dict with a np.ndarray() as required without acutally performing any pre-processing on the initial text. However, then it told me that name 'numpy' is not defined…
EDIT3: Uninstalling and installing dill and multiprocess did not help. Also, dill.dumps() works fine on both, the tokenizer and dataset - or least it does not throw and error but returns a lot of cryptic stuff.
EDIT4: I can’t reproduce this error in a colab environment. Likely something local?