There are some formal methods and rules of thumb for avoiding trouble in this regard. Take a look at bayesian and entropy-based maximization strategies. Hyperparameter optimization can be looked at using either strategy. Also take a look at <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Minimum_description_length" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Minimum_description_length</a><p>There&#x27;s definitely a lot of theory, but it hasn&#x27;t yet been turned into readily usable R libraries.