I would recommend getting the starting learning rate right at first, then do some fine tuning with a learning rate scheduler. For other parameters, I would suggest copying them from the professionals (check a few papers and github codes)  when starting. After a while you will be able to roughly estimate how your parameters should be from experience.
Unfortunately, what I’m describing is the ideal situation. This depends a lot on what you are doing and your resources. If your model takes a long time to train or you don’t have the resources to train it quickly (contrarily to popular belief, not every researcher has >100 gpus ), you may not be able to do a lot fine tuning with your hyperparameters.