This should work:
image_arr = []

for i in range(len(train_dataset)):
    image_arr.append(to_tensor(train_dataset[i]['image']))

image_arr = [x.unsqueeze_(0) for x in image_arr]
image_tens = torch.cat(image_arr, dim=0)

print(image_tens.mean((0,1,2)))

Basically, if you have a list of tensors, each of shape [height, width, channels], then you can add one more dimension to each tensor using unsqueeze(), to get each tensor in the list to be of shape [1, height, width, channels], then you can concatenate all those tensors to get a bigger tensor of size [len(image_arr), height, width, channels]. Now if the whole object is a tensor, it’s easy to calculate mean.