I am working on a CNN which transforms black and white images to color images. I encountered a problem though. The network is basically “averaging out” all input images into this blurry mush you can see on the pic below:

The loss doesn’t decrease any further regardless of my learning rate setting. This is my average loss per epoch of the training set (red line), and the validation set (blue line):

ai2.jpg731×346 20.4 KB

I am using around 3800 images in my training set (randomly flipped vertically, horizontally and randomly rotated up to 180 degrees). Here you can see a few input vs expected output images:

I am using MSE and Adam as my loss function and optimizer, respectively.
Could you please help me out with this? I am not sure what to tweak anymore .