It is possible but does your loss function works the same way for all the channels? Or do you have any specific loss for the 4th channel? I mean don’t you need the edge map somewhere in your calculation?
If it’s the same for all channels then I think adding a conv2d layer should solve your problem.