When looking into Hyperparameter optimization you could have a look into “AutoML” techniques which automate the optimization of hyperparamters. One example within PyTorch is Auto-PyTorch, you can find their github repo here: GitHub - automl/Auto-PyTorch: Automatic architecture search and hyperparameter optimization for PyTorch 1 !