Yes all of these are discrete problems and that’s why you see very little work actually learning such things.
People usually do grid-search to try different architectures / kernel size etc… Because there are not much more options.
One line of work that went around this (with limited success though) is that work on differentiable computers that you can look at.