Can I enforce monotonically increasing neural net outputs (min, mean, max)? Hi I'm using a DL model (TensorFlow) to predict daily minimum, mean, and maximum values of a target dataset. I was thinking that the model would have 3 outputs for each day, (min, mean, max).
Is there a clean way to enforce the correct order of these (i.e., min<mean<max)? I can add a penalty to encourage the model to train that way, but that seems like a bit of a work-around.

I would suggest you are better off investigating quantile regression.
see eg
https://towardsdatascience.com/deep-quantile-regression-in-tensorflow-1dbc792fe597.
What I believe you want is to predict the mean and limits of your distribution. at each x value.  using the minimum and maximum as targets will not give you what you want. instead you will have the expected value of the minimum/maximum at each input (this is what MSE produces).
eg input is x




id
x
y_mean
y_max




1
1
1
1


2
1
1
3


3
1
1
5




then the prediction for y_max for input x=1 will be 3, not 5.
if instead you predict the quantiles you will get eg 10 percentile, 50 percentile and 90 percentile of your distribution.
(an alternative simpler way is to continue with mse,but predict the mean and squared error.  that gives you the variance and you can eg estimate the percentiles making a normal approximation)
