OpenAI Five
So as someone working in reinforcement learning who has used PPO a fair bit, I find this quite disappointing from an algorithmic perspective.<p>The resources used for this are almost absurd and my suspicion is, especially considering [0], that this comes down to an incredibly expensive random search in the policy space. Or rather, I would want to see a fair bit of analysis to be shown otherwise.<p>Especially given all the work in intrinsic motivation, hierarchical learning, subtask learning, etc, the sort of intermediate summary of most of these papers from 2015-2018 is that so many of these newer heuristics are too brittle&#x2F;difficult to make work, so we resort to slightly-better-than brute force.<p><a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1803.07055" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1803.07055</a>