Solved by rasbt in post #2 
              
              
                You don’t need to do anything special to perform bayesian optimization for your hyperparameter tuning when using pytorch. You could just setup a script with command line arguments like --learning_rate, --num_layers for the hyperparameters you want to tune and maybe have a second script that calls th…