Can I enforce monotonically increasing neural net outputs (min, mean, max)? Hi I'm using a DL model (TensorFlow) to predict daily minimum, mean, and maximum values of a target dataset. I was thinking that the model would have 3 outputs for each day, (min, mean, max).
Is there a clean way to enforce the correct order of these (i.e., min<mean<max)? I can add a penalty to encourage the model to train that way, but that seems like a bit of a work-around.

It may be infeasible to formally check the KKT conditions for your optimization problem, however you can still try encoding your inequality constraints as if the conditions hold. Then it is a matter of checking whether the training behaves nicely in practice.
If you're unfamiliar with encoding constraints into an objective function in the way that KKT states, see Lagrange multipliers for a first example with equality constraints. Then I recommend you look at encoding inequality constraints (see example) in a similar fashion.
Once you have the mathematical expression in hand, you'll need to implement it in Tensorflow. You can build your own loss function class by inheriting from the tf.keras.losses.Loss base class.
