Gaussian process on hyperparameter tuning I feel it is kind of circular to use GP for hyperparameter tuning, since GP has its own hyperparameters. Or is it the case that GP typically has less number of hyperparameters than the model we want to tune (say NNs), which mitigates the issue somewhat?

You are correct Gaussian process has it's own hyperparameters. But same is true for every other hyperparameter tuning algorithm you would use. If you use grid search, you need to decide on the grid of points to use, if you want to do random search, you need to decide on the distributions used for sampling the parameters, etc.
The good news is that in many cases, the models we use are not that sensitive to the hyperparameters (it doesn't matter that much if the learning rate is 0.01, 0.013, or 0.014, but matters if it's closer to 0.0001), so it is not that important for the algorithms that we use for hyperparameter tuning to be that precise.
