You could try to iterate once over the data loader and check the size of the inputs, just to be sure?
This might help you locate the error more precisely (e.g. if all images have 4 channels, or if they have 3 channels up to this point and are modified later on…)