Thanks to @user_123454321 we have an answer.
Indeed, __len__ doesn’t influence how an iterable is converted to a list or iterated over in for loops. In fact, you have to implement and IndexError which signals the end of the iteration. This is also described in the __getitem__ docs
So, the correct implementation is as follows:
from torch.utils.data import Dataset

class NumbersDataset(Dataset):
  def __init__(self):
    self.samples = [1,2,3,4,5]

  def __len__(self):
    return 3

  def __getitem__(self, idx):
    if idx>2: 
      raise IndexError
    return self.samples[idx]


dataset = NumbersDataset()

len(dataset), len(list(dataset))
>>> 3, 3

list(dataset)
>>> [1, 2, 3]