As long as n_jobs=1 in hyperparameter_search I do not think the search is in parallel. If you are still running into issues you may want to decrease the batch size and use gradient accumulation instead.