Hi @witl
First day on the forum because of your post. This is something I am experienced with and interested in.

I wonder when it is recommended (or mandatory) to use metaheuristics instead of gradient descent? I believe that it is related to the cases where you don’t have an input, output, and target. If you dont have a target you cannot calculate the loss, and then you cannot perform gradient descent. Is it right?

Good question, I will look more into this. Learning the theory along the journey!

Do you have any example of problems where the model can only be trained by the usage of metaheuristics?

Any test subjects will work. GLM’s, Deep learning, image-video-text analysis, reinforcement learning.

For me, it looks like there is much more freedom in the metaheuristic case. Maybe it is because I am more used to formulate problems like that. Do you have any experience with that?

Yeah, metaheuristics are dope AF.