@yjernite
this part has some issues in my jupiter
def tokenize_function(examples):
    return tokenizer(examples["text"])

when called from
tokenized_datasets = datasets.map(tokenize_function, batched=True, 
num_proc=4, remove_columns=["text"])

File “”, line 2, in tokenize_function
NameError: name ‘tokenizer’ is not defined