segum:

was thinking of trying leaky relu/initialization of biases of conv layers to 0.1/directly removing the last relu.


Hm, what loss function do you use? Can you share the code snippet/line that implements and computes the loss?