This line here 849 in torchvision is reason for 3 channel loading. You can write your own function to load image like at this line and pass to loader parameter of ImageFolder.
Just a headsup: If you need to load MNIST, there’s a torch vision dataset for that.
Hope this helps.