What kind of hyper-parameter would you like to tune?
Usually the hyper-parameters of the training routine are e.g. the learning rate, weight decay etc.
You could tune it using your training and validation loss, but I’m not sure I understand the connection to the loss function completely.