@FelipeVW
In my opinion, you are 75% right, In the case of something like a CNN, you can scale down your model procedurally so it takes much less time to train, THEN do hyperparameter tuning.  This paper 226 found that a grid search to obtain the best accuracy possible, THEN scaling up the complexity of the model led to superior accuracy. Probably would not work for all cases, but definitely a good application for grid searches.