You just need to change the first layer.
There are plenty of solutions already discussed here.
  
    
    
    Modify ResNet or VGG for single channel grayscale 
  
  
    Hi, 
I’m working on infrared data which I convert to grayscale 64x64 (although I can use other sizes, but usually my GPU runs out of memory). I used VGG11 but I manually recreated the architecture in order to use it, which is a very cumbersome and error prone task. 
I was wondering if there is an easier way to modify VGG19 or ResNet architectures in a fast and simpler way to use my 64x64 single channel input, and if yes, would that make sense since those models are fine-tuned for 3 channel RGB? 
…
  


  
    
    
    Grayscale images for resenet and deeplabv3 vision
  
  
    How can I modify a resnet or VGG network to use grayscale images. I am loading the network the following way 
m=torchvision.models.segmentation.fcn_resnet50(pretrained=False, progress=True, num_classes=2, aux_loss=None)

Is there some way I can tweak this model after loading it? 
Thanks