It seems that the error is raised due to a shape mismatch between the model output and the target.
Could you check their shapes before feeding them to the criterion and also post the code how you are creating these tensors?
The other issue regarding the shape mismatch for RGB and grayscale images, is that it seems you are defining in_features=1024 in your first layer. While this would fit the input dimension of a grayscale 32*32=1024 image, it will fail for a 3*32*32 image, so you would have to increase the number of input features by 3.