You could check for the number of channels and based on this chose a separate model input layer.
However, I wouldn’t apply this method.
The straightforward solution would be to load your images and convert them to RGB:
image = Image.open(path).convert('RGB')

You can add this line to your Dataset's __getitem__ method.