train_dataset.indices is a list with indices. So when I call valid_dataset.dataset.data I get the whole data from the dataset. In order to only get only those that belong to the Subset I can select them with indexing train_dataset.dataset.data[train_dataset.indices]. My data is a tensor, but it could also be a numpy array.
You can also loop over it I think:
current_mean = 0.0
for i in train_dataset.indices:
    current_mean += train_dataset.dataset.data[i]
current_mean /= len(train_dataset.indices)

Hope this helps!