Hi,
I tried nlp but I got an error when I used it on a TPU node (https://github.com/huggingface/nlp/issues/532 11). Does the way I use it seem fine?
I modified line 131 in the original run_language_modeling.py 15 as follows:
# line 131: return LineByLineTextDataset(tokenizer=tokenizer, file_path=file_path, block_size=args.block_size)
dataset = load_dataset("text", data_files=file_path, split="train")
dataset = dataset.map(lambda ex: tokenizer(ex["text"], add_special_tokens=True,
                                        truncation=True, max_length=args.block_size), batched=True)
dataset.set_format(type='torch', columns=['input_ids'])
return dataset