Solved by nielsr in post #2 
              
              
                You can update your function, to add both the encoded text, as well as the tokens: 
def tokenize_function_testing(examples):
     encoding = bert_tokenizer(examples["sentences"], padding="max_length", max_length=32, truncation=True)
     # add tokens
     encoding["tokens"] = bert_tokenizer.convert_…