Indeed I ran a timing test comparing the two. The code is as follows:
import time
start_time = time.time()

for i in range(0, dataset.nitems-200, 200):
        batch = dataset[i,i+200]
        print(i)
        print(batch[1])
        print(batch[2])

dataset.close()
print("--- %s seconds ---" % (time.time() - start_time))

And for the dataloader
import time
start_time = time.time()

dataloader = DataLoader(dataset, batch_size=200, shuffle=False, num_workers=16,pin_memory=False,collate_fn=text_dataset_collate)

for i, batch in enumerate(dataloader):
       print(i)
       print(batch[1])
       print(batch[2])

dataset.close()
print("--- %s seconds ---" % (time.time() - start_time))

Custom batching takes
--- 2.1859922409057617 seconds ---

The dataloader takes
--- 19.42865824699402 seconds ---

I know this is not quite perfect, because of printing as operation (is this significant) and also the dataset is rather small, only 200k sequences.
But the difference is staggering, even though the dataloader process runs on 16 threads, and the custom batching is simply a loop?
For other batch sizes, differences are even larger. In particular, the dataloader hovers around 20 seconds, decreasing slightly with batch sizes, but the custom loop goes down to things like 0.05 seconds!
Profiling shows that the main difference are the dataloader functions, next and get_data. So it really seems that loading data step by step is inefficient
What am I doing wrong?
This can not be right