It makes sense that it might cause a problem, though     transforms.Grayscale(num_output_channels=3) should deal with channels.
This change results in a quite similar error:
TypeError: Traceback (most recent call last):
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in 
samples = collate_fn([dataset[i] for i in batch_indices])
File “”, line 37, in getitem
x = self.svhn_transform(x)
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 49, in call
img = t(img)
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 175, in call
return F.resize(img, self.size, self.interpolation)
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/functional.py”, line 189, in resize
raise TypeError(‘img should be PIL Image. Got {}’.format(type(img)))
TypeError: img should be PIL Image. Got <class ‘numpy.ndarray’>