overallperformance.8AcknowledgementsThankstoRockyDuan,PeterChen,andothersatOpenAIforinsightfulcomments.8References[Bel+15]M.Bellemare,Y.Naddaf,J.Veness,andM.Bowling.“Thearcadelearningenviron-ment:Anevaluationplatformforgeneralagents”.In:Twenty-FourthInternationalJointConferenceonArtiﬁcialIntelligence.2015.[Bro+16]G.Brockman,V.Cheung,L.Pettersson,J.Schneider,J.Schulman,J.Tang,andW.Zaremba.“OpenAIGym”.In:arXivpreprintarXiv:1606.01540(2016).[Dua+16]Y.Duan,X.Chen,R.Houthooft,J.Schulman,andP.Abbeel.“BenchmarkingDeepReinforcementLearningforContinuousControl”.In:arXivpreprintarXiv:1604.06778(2016).[Hee+17]N.Heess,S.Sriram,J.Lemmon,J.Merel,G.Wayne,Y.Tassa,T.Erez,Z.Wang,A.Eslami,M.Riedmiller,etal.“EmergenceofLocomotionBehavioursinRichEnvi-ronments”.In:arXivpreprintarXiv:1707.02286(2017).[KL02]S.KakadeandJ.Langford.“Approximatelyoptimalapproximatereinforcementlearn-ing”.In:ICML.Vol.2.2002,pp.267–274.[KB14]D.KingmaandJ.Ba.“Adam:Amethodforstochasticoptimization”.In:arXivpreprintarXiv:1412.6980(20
PPO