Same trick as I ended up using (had no issues with overfitting, but I guess that depends on the use-case). In the mean time I also opened an issue on github to make it possible to normalize on any axis.