ElLoboLoco:

At first I thought to initialize the weights of the output layer to zero but it will prevent it from learning


It won’t, this only applies to hidden layers (that generate “artifical” features). For output layer, loss (eq.:  distance to target) provides a non-zero gradient.