Generally capable agents emerge from open-ended play
Agents trained in simulation like this often flail about seemingly randomly, and when they achieve their goals it seems almost accidental. Rather than this being some kind of limitation of the learning algorithm, I think it might be the optimal strategy, and  humans would behave that way too if there was no such thing as fatigue or pain.<p>If we want agents to behave more realistically and move with more apparent intention we need cost functions that include a "pain" and/or fatigue term to penalize flailing behavior. But that adds hyperparameters that need to be carefully tuned to balance penalties with rewards, otherwise training will be unstable or simply fail.<p>I wonder if there's a principled way to determine an appropriate cost function without manual tuning. Did evolution serve as the "manual" optimizer that generated a precisely tuned cost function for the human brain? Or did evolution discover a generally applicable method for automatically generating cost functions, which the brain then applies to whatever input it gets?