In nlp 0.4.0 You can’t use a map function that returns pytorch tensors. You can fix it my removing the “return_tensors=‘pt’” in your tokenize function. This is because the dataset is saved in Arrow format.
It’s only recently that we’ve added the support for torch/tf tensors inputs (see changes here 41) and it will be will be available in the next release.