So I’m not sure if I understand correctly what you have in mind, but a few quick comments that shape my intuition around this:

As you write, the Lagrange multiplier helps turn a constrained problem into an unconstrained one. While it does so, however, it turns the (say) minimization problem into a saddle point problem. (Because you want the min_{variables} max_{Lagrange multipliers} L, and whenever your constraint is not satisfied, you get infinity in the max part, so it will be irrelevant to the min (if an admissible point exists).)
If you look at regularization, typically, you’ll have a weight for the regularization term as a hyperparameter. Now, if you naïvely included that in your optimization while enforcing positivity, you would just drive that to 0, because, obviously no penalty is cheaper than a penalty.
You can think of tuning as “optimizing hyperparameters” (and the Bayesian crowd will have thoughts on integrating over hyperparameters). This can be done if you set an objective separate from the “inner” optimization objectives, e.g. (as for manual tuning) loss on a validation set. This often goes under the label “meta-learning”.
The perhaps best-studied regularization method is Tikhonov regularization (leading to “ridge regression”) 3. As the Wikipedia page notes, the optimal regularization parameter is typically unknown. It does offer some heuristics / discussion of helpful interpretations.
Yarin Gal, in his thesis 2, has a section on optimizing the dropout probability in the context of variational Bayesian inference (Section 6.4 ELBO correlation with test log likelihood), where he reports mixed results and offers some hypothesis for possible causes. I must admit I did not follow whether further research exists on any of these hypothesis.

Best regards
Thomas