Took some time to get back to you. First, thanks you for helping me look into it.
The gradient of all my layers is zero. I don’t understand why…
 print('\nGrad : \n')
 print('conv0', self.model.conv0.weight.grad)
 print('conv3', self.model.conv3.weight.grad)
 print('linear0', self.model.linear0.weight.grad)
 print('linear1', self.model.linear1.weight.grad)


>
Grad : 

conv0 tensor([[[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]]])
conv3 tensor([[[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]],


        [[[0., 0., 0.],
          [0., 0., 0.],
          [0., 0., 0.]]]])
linear0 tensor([[0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0.]])
linear1 tensor([[0., 0., 0., 0.]])

Could it be that the operations at the end between the sigmoid and the int values vmin and vmax could be at fault here and if yes why ? Btw I tried to set vmin and vmax to be torch.FloatTensor scalars that I also set to the same device but it did no good. you can find bellow an example of how I set those values:
vmin = torch.tensor(0) - 21
vmin = vmin.type(torch.FloatTensor).to(device).requires_grad_(True)