Dare "corpo" all'IA: il nostro futuro con i robot intelligenti
L'intersezione tra IA e robotica apre nuovi scenari applicativi, in cui l'intelligenza dei robot va oltre la semplice produzione di testi e ...
pubblicato
Un’altra tecnica è la sim-to-real reinforcement learning. Gli algoritmi del robot si esercitano, con tentativi ed errori, in una realtà simulata, miliardi di volte, prima di passare a quella fisica. Così funziona il robot umanoide Digit. Ne parla un paper <anchor>pubblicato</anchor> qualche giorno fa su Science Robotics e frutto di una collaborazione tra università dell’Oregon e della California, Berkeley.
Real-world humanoid locomotion with reinforcement learning | Science Robotics
Real-world humanoid locomotion with reinforcement learning \| Science Robotics

Advertisement

- [news](/news)
- [careers](/careers)
- [commentary](/commentary)
- [Journals](/journals)

[![Science](/pb-assets/images/styleguide/logo-dark-1672180581427.svg)![Science](/pb-assets/images/styleguide/logo-1672180580750.svg)](/)

- [Search](#header-quick-search-wrapper "Search")
- [Log in](/action/ssostart?redirectUri=%2Fdoi%2F10.1126%2Fscirobotics.adi9579)

- [Become A Member](https://promo.aaas.org/science/join/?CTC=SMHPJN "")


[![journal-menu-img](/cms/asset/5f9925d7-2fdb-4db7-91d0-4282ee23d988/science.2024.385.issue-6714.cover.gif)\
science](/journal/science "Science")

[![journal-menu-img](/cms/asset/0c0b6bd8-0958-4f95-a0d3-5f55a891d9a9/sciadv.2024.10.issue-37.cover.gif)\
science advances](/journal/sciadv "Science")

[![journal-menu-img](/cms/asset/945cb50a-a2a9-48a9-9c41-2f698f8cc1ea/sciimmunol.2024.9.issue-99.cover.gif)\
science immunology](/journal/sciimmunol "Science")

[![journal-menu-img](/cms/asset/20335fd0-680b-4e2f-97d1-9cfebcc7ea33/scirobotics.2024.9.issue-94.cover.gif)\
science robotics](/journal/scirobotics "Science")

[![journal-menu-img](/cms/asset/a96446a7-fa20-4d3f-b74b-8e18c76e36f9/signaling.2024.17.issue-853.cover.gif)\
science signaling](/journal/signaling "Science")

[![journal-menu-img](/cms/asset/aba85a83-5ef9-427a-a7e9-5c6b692132a1/stm.2024.16.issue-764.cover.gif)\
science translational medicine](/journal/stm "Science")

[![journal-menu-img](/pb-assets/images/styleguide/spj-cover-1695403298717.png)\
science partner journals](https://spj.science.org/ "Science Partner Journals")

Quick Search anywhere

Enter Search Term

Quick Search in Journals

Enter Search Term

Quick Search in Journals

Enter Search Term

Quick Search in Journals

Enter Search Term

Quick Search in Journals

Enter Search Term

Quick Search in Journals

Enter Search Term

Quick Search in Journals

Enter Search Term

Searching:

Anywhere

[Anywhere](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-0) [Science](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-1) [Science Advances](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-2) [Science Immunology](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-3) [Science Robotics](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-4) [Science Signaling](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-5) [Science Translational Medicine](#quick-search-form-9ea9b667-c3ce-40d1-870b-f7e001bbac01-6)

[Advanced Search](/search/advanced) Search

###### Trending Terms:

- [cancer](/action/doSearch?AllField=cancer)
- [climate](/action/doSearch?AllField=climate)
- [artificial intelligence](/action/doSearch?AllField=artificial intelligence)
- [postdoc](/action/doSearch?AllField=postdoc)
- [aging](/action/doSearch?AllField=aging)

[Log In](/action/ssostart?redirectUri=/doi/10.1126/scirobotics.adi9579)

[Become A Member](/action/clickThrough?id=501021&url=https%3A%2F%2Fpromo.aaas.org%2Fscience%2Fjoin%2F%3FCTC%3DSMLDJN&loc=%2Fdoi%2F10.1126%2Fscirobotics.adi9579&pubId=42042443&placeholderId=501021&productId=501021)

[Donate](/news/donate?intcmp=sitemenu-donate&utm_id=recB0g01G48yiGSBD)

Quick Search anywhere

Enter Search Term

[science.org](/)

- [Custom publishing](/custom-publishing)
- [newsletters](/topic/article-type/scienceadviser?intcmp=menu-adviserfeed&utm_id=recI11u4srAIiGNLZ)
- [collections](/collections)
- [videos](/videos)
- [podcasts](/podcasts)
- [blogs](/blogs)
- [visualizations](/content/page/visualizations)
- [prizes and awards](/content/page/prizes-and-awards)

- [authors & reviewers](/content/page/contributing-science-family-journals)
- [librarians](/content/page/librarian-portal)
- [advertisers](https://advertising.sciencemag.org/)
- [about](/content/page/aboutus)
- [help](/content/page/help)

- [Facebook](https://www.facebook.com/ScienceMagazine "Facebook")
- [Share on X](https://twitter.com/sciencemagazine "Share on X")
- [Instagram](https://www.instagram.com/ScienceMagazine "Instagram")
- [YouTube](https://www.youtube.com/user/ScienceMag "YouTube")
- [RSS feeds](/content/page/email-alerts-and-rss-feeds "RSS feeds")
- [WeChat](https://mp.weixin.qq.com/s?__biz=MzI3NDY3NzQ2Mg==&mid=100002815&idx=1&sn=2949c025a553ac718b9612a0473b9f60&chksm=6b1120465c66a9508b01eaef1589b15d440e50b189106c8c594de8c6471f696a978de952fb15&mpshare=1&scene=1&srcid=0716JJQ5V4cKbgMMsya2MQ0n&sharer_sharetime= "WeChat")

![AAAS Logo](/pb-assets/images/styleguide/logo-1672180580750.svg)

- [Terms of Service](/content/page/terms-service)
- [Privacy Policy](/content/page/privacy-policy)
- [Accessibility](/content/page/accessibility)

[![logo](/pb-assets/images/logos/scirobotics-logo-1620488350107.svg)](/journal/scirobotics)

- [Current Issue](/toc/scirobotics/current)
- [Archive](/loi/scirobotics)
- About

[About Science Robotics](/content/page/science-robotics-mission-and-scope) [Mission & Scope](/content/page/science-robotics-mission-and-scope) [Editors & Advisory Boards](/content/page/science-robotics-editorial-and-advisory-boards) [Editorial Policies](/content/page/science-journals-editorial-policies) [Information for Authors](/content/page/science-robotics-information-authors) [Information for Reviewers](/content/page/science-robotics-information-reviewers) [Staff](/content/page/science-robotics-staff) [Journal Metrics](/content/page/journal-metrics-overview) [Frequently Asked Questions](/content/page/science-robotics-faq) [Contact Us](/content/page/contact-science-robotics) [Free Sample Articles](/content/page/science-robotics-free-sample-articles) [Call for Papers](/content/page/calls-papers) [TOC Alerts and RSS Feeds](/content/page/email-alerts-and-rss-feeds)


- [Submit manuscript](https://cts.sciencemag.org/)
- [More](About)


[GET OUR E-ALERTS](/action/showPreferences?menuTab=Alerts)

[Home](https://www.science.org/ "Home") [Science Robotics](/journal/scirobotics "Science Robotics") [Vol. 9, No. 89](/toc/scirobotics/9/89 "Vol. 9, No. 89") Real-world humanoid locomotion with reinforcement learning

[Back To Vol. 9, No. 89](/toc/scirobotics/9/89 "Back To Vol. 9, No. 89")

No access

Research Article

ROBOT LOCOMOTION

Share on

# Real-world humanoid locomotion with reinforcement learning

[IlijaRadosavovic](#con1) [https://orcid.org/0009-0004-9200-5980](https://orcid.org/0009-0004-9200-5980) [\[email protected\]](/cdn-cgi/l/email-protection#5b323732313a1b393e29303e373e22753e3f2e643838662f23323a341b393e29303e373e22753e3f2e7d3a362b603838663932303e21333a353c1b393e29303e373e22753e3f2e), [TeteXiao](#con2) [https://orcid.org/0009-0006-6440-2393](https://orcid.org/0009-0006-6440-2393) [\[email protected\]](/cdn-cgi/l/email-protection#274e4b4e4d46674542554c424b425e094243521844441a535f4e4648674542554c424b425e0942435201464a571c44441a454e4c425d4f464940674542554c424b425e09424352), \[...\], [BikeZhang](#con3) [https://orcid.org/0000-0002-2652-4318](https://orcid.org/0000-0002-2652-4318) [\[email protected\]](/cdn-cgi/l/email-protection#741d181d1e15341611061f1118110d5a1110014b171749000c1d151b341611061f1118110d5a111001521519044f171749161d1f110e1c151a13341611061f1118110d5a111001), [TrevorDarrell](#con4), \[...\], [JitendraMalik](#con5) [https://orcid.org/0000-0003-3695-1580](https://orcid.org/0000-0003-3695-1580), and [KoushilSreenath](#con6) [https://orcid.org/0000-0002-5346-3637](https://orcid.org/0000-0002-5346-3637)+3 authors+1 authorsfewer [Authors Info & Affiliations](#tab-contributors)

Science Robotics

17 Apr 2024

Vol 9, Issue 89

[DOI: 10.1126/scirobotics.adi9579](https://doi.org/10.1126/scirobotics.adi9579)

[**PREVIOUS ARTICLE** \
\
Learning robust autonomous navigation and locomotion for wheeled-legged robots\
\
Previous](/doi/10.1126/scirobotics.adi9641 "Learning robust autonomous navigation and locomotion for wheeled-legged robots") [**NEXT ARTICLE** \
\
Locomotion as manipulation with ReachBot\
\
Next](/doi/10.1126/scirobotics.adi9762 "Locomotion as manipulation with ReachBot")

[Notifications](/action/addCitationAlert?doi=10.1126%2Fscirobotics.adi9579 "Notifications")[Bookmark](/personalize/addFavoritePublication?doi=10.1126%2Fscirobotics.adi9579 "Bookmark")

[CITE](#tab-citations "CITE")

[CHECK ACCESS](#core-collateral-purchase-access "CHECK ACCESS")

- Contents

- [Editor’s summary](#editor-abstract)
- [Abstract](#abstract)
- [Supplementary Materials](#supplementary-materials)
- [REFERENCES AND NOTES](#bibliography)

- - [Information & Authors](#core-collateral-info)
- [Metrics & Citations](#core-collateral-metrics)
- [Check Access](#core-collateral-purchase-access)
- [References](#core-collateral-references)
- [Media](#core-collateral-media)
- [Tables](#core-collateral-tables)
- [Share](#core-collateral-share)

## Editor’s summary

The ability of robots to navigate adaptively and robustly in varying terrain increases their chances of success when deployed in the real world. However, stable locomotion of full-size bipedal humanoid robots creates a challenge from a controls perspective. Radosavovic _et al_. developed a reinforcement learning approach for controlling locomotion of a humanoid robot, Digit. They trained their model in simulation and subsequently deployed it into the real-world zero-shot and showed the potential for robust locomotion on various indoor and outdoor environments. The robot could exhibit natural and adaptive walking behaviors, including an emergent arm-swing motion, and adapt to external perturbations. —Amos Matsiko

## Abstract

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labor shortages in factories, assist elderly at home, and colonize new planets. Although classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesized that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in context, without updating its weights. We trained our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed it to the real-world zero-shot. Our controller could walk over various outdoor terrains, was robust to external disturbances, and could adapt in context.

## Get full access to this article

View all available purchase options and get full access to this article.

[CHECK ACCESS](#core-collateral-purchase-access)

ALREADY A SUBSCRIBER OR AAAS MEMBER? Sign in as an [individual](/action/showLogin?redirectUri=%2Fdoi%2F10.1126%2Fscirobotics.adi9579) or via your [institution](/action/ssostart?redirectUri=%2Fdoi%2F10.1126%2Fscirobotics.adi9579)

## Supplementary Materials

### The PDF file includes:

Text S1 and S2

Tables S1 to S4

- [Download](/doi/suppl/10.1126/scirobotics.adi9579/suppl_file/scirobotics.adi9579_sm.pdf)
- 275.78 KB

### Other Supplementary Material for this manuscript includes the following:

Movies S1 to S6

- [Download](/doi/suppl/10.1126/scirobotics.adi9579/suppl_file/scirobotics.adi9579_movies_s1_to_s6.zip)
- 174.10 MB

## REFERENCES AND NOTES

1

I. Kato, Development of WABOT 1, in _Biomechanism_ (University of Tokyo Press, 1973).

[Google Scholar](https://scholar.google.com/scholar?q=I.+Kato%2C+Development+of+WABOT+1%2C+in+Biomechanism+%28University+of+Tokyo+Press%2C+1973%29.)

2

K. Hirai, M. Hirose, Y. Haikawa, T. Takenaka, The development of Honda humanoid robot, in _IEEE International Conference on Robotics and Automation (ICRA)_ (IEEE, 1998), vol. 2, pp. 1321–1326.

[Google Scholar](https://scholar.google.com/scholar?q=K.+Hirai%2C+M.+Hirose%2C+Y.+Haikawa%2C+T.+Takenaka%2C+The+development+of+Honda+humanoid+robot%2C+in+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29+%28IEEE%2C+1998%29%2C+vol.+2%2C+pp.+1321%E2%80%931326.)

3

G. Nelson, A. Saunders, N. Neville, B. Swilling, J. Bondaryk, D. Billings, C. Lee, R. Playter, M. Raibert, Petman: A humanoid robot for testing chemical protective clothing. _J. Robot. Soc. Jpn._ **30**, 372–377 (2012).

[Crossref](https://doi.org/10.7210/jrsj.30.372)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Petman%3A+A+humanoid+robot+for+testing+chemical+protective+clothing&author=G.+Nelson&author=A.+Saunders&author=N.+Neville&author=B.+Swilling&author=J.+Bondaryk&author=D.+Billings&author=C.+Lee&author=R.+Playter&author=M.+Raibert&publication_year=2012&journal=J.+Robot.+Soc.+Jpn.&pages=372-377&doi=10.7210%2Fjrsj.30.372)

4

O. Stasse, T. Flayols, R. Budhiraja, K. Giraud-Esclasse, J. Carpentier, J. Mirabel, A. Del Prete, P. Souères, N. Mansard, F. Lamiraux, J. P. Laumond, L. Marchionni, H. Tome, F. Ferro, TALOS: A new humanoid research platform targeted for industrial applications, in _IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)_ (IEEE, 2017), pp. 689–695.

[Google Scholar](https://scholar.google.com/scholar?q=O.+Stasse%2C+T.+Flayols%2C+R.+Budhiraja%2C+K.+Giraud-Esclasse%2C+J.+Carpentier%2C+J.+Mirabel%2C+A.+Del+Prete%2C+P.+Sou%C3%A8res%2C+N.+Mansard%2C+F.+Lamiraux%2C+J.+P.+Laumond%2C+L.+Marchionni%2C+H.+Tome%2C+F.+Ferro%2C+TALOS%3A+A+new+humanoid+research+platform+targeted+for+industrial+applications%2C+in+IEEE-RAS+17th+International+Conference+on+Humanoid+Robotics+%28Humanoids%29+%28IEEE%2C+2017%29%2C+pp.+689%E2%80%93695.)

5

M. Chignoli, D. Kim, E. Stanger-Jones, S. Kim, The MIT humanoid robot: Design, motion planning, and control for acrobatic behaviors, in _IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids)_ (IEEE, 2021), pp. 1–8.

[Google Scholar](https://scholar.google.com/scholar?q=M.+Chignoli%2C+D.+Kim%2C+E.+Stanger-Jones%2C+S.+Kim%2C+The+MIT+humanoid+robot%3A+Design%2C+motion+planning%2C+and+control+for+acrobatic+behaviors%2C+in+IEEE-RAS+20th+International+Conference+on+Humanoid+Robots+%28Humanoids%29+%28IEEE%2C+2021%29%2C+pp.+1%E2%80%938.)

6

M. H. Raibert, _Legged Robots That Balance_ (MIT Press, 1986).

[Google Scholar](https://scholar.google.com/scholar?q=M.+H.+Raibert%2C+Legged+Robots+That+Balance+%28MIT+Press%2C+1986%29.)

7

S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, H. Hirukawa, The 3D linear inverted pendulum mode: A simple modeling for a biped walking pattern generation, in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2001).

[Google Scholar](https://scholar.google.com/scholar?q=S.+Kajita%2C+F.+Kanehiro%2C+K.+Kaneko%2C+K.+Yokoi%2C+H.+Hirukawa%2C+The+3D+linear+inverted+pendulum+mode%3A+A+simple+modeling+for+a+biped+walking+pattern+generation%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2001%29.)

8

E. R. Westervelt, J. W. Grizzle, D. E. Koditschek, Hybrid zero dynamics of planar biped walkers. _IEEE Trans Automat Contr_ **48**, 42–56 (2003).

[Crossref](https://doi.org/10.1109/TAC.2002.806653)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000180424600004)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Hybrid+zero+dynamics+of+planar+biped+walkers&author=E.+R.+Westervelt&author=J.+W.+Grizzle&author=D.+E.+Koditschek&publication_year=2003&journal=IEEE+Trans+Automat+Contr&pages=42-56&doi=10.1109%2FTAC.2002.806653)

9

S. Collins, A. Ruina, R. Tedrake, M. Wisse, Efficient bipedal robots based on passive-dynamic walkers. _Science_ **307**, 1082–1085 (2005).

[Crossref](https://doi.org/10.1126/science.1107799)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/15718465/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000227197300040)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Efficient+bipedal+robots+based+on+passive-dynamic+walkers&author=S.+Collins&author=A.+Ruina&author=R.+Tedrake&author=M.+Wisse&publication_year=2005&journal=Science&pages=1082-1085&doi=10.1126%2Fscience.1107799&pmid=15718465)

10

Y. Tassa, T. Erez, E. Todorov, Synthesis and stabilization of complex behaviors through online trajectory optimization, in _IEEE/RSJ International Conference on Intelligent Robots and Systems_ (IEEE, 2012), pp. 4906–4913.

[Google Scholar](https://scholar.google.com/scholar?q=Y.+Tassa%2C+T.+Erez%2C+E.+Todorov%2C+Synthesis+and+stabilization+of+complex+behaviors+through+online+trajectory+optimization%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IEEE%2C+2012%29%2C+pp.+4906%E2%80%934913.)

11

S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, R. Tedrake, Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. _Auton Robots_ **40**, 429–455 (2016).

[Crossref](https://doi.org/10.1007/s10514-015-9479-3)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000371241500002)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Optimization-based+locomotion+planning%2C+estimation%2C+and+control+design+for+the+atlas+humanoid+robot&author=S.+Kuindersma&author=R.+Deits&author=M.+Fallon&author=A.+Valenzuela&author=H.+Dai&author=F.+Permenter&author=T.+Koolen&author=P.+Marion&author=R.+Tedrake&publication_year=2016&journal=Auton+Robots&pages=429-455&doi=10.1007%2Fs10514-015-9479-3)

12

J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, S. Kim, Dynamic locomotion in the MIT Cheetah 3 through convex model-predictive control, in _IEEE/RSJ International Conference on Intelligent Robots And Systems (IROS)_ (IEEE, 2018), pp. 1–9.

[Google Scholar](https://scholar.google.com/scholar?q=J.+Di+Carlo%2C+P.+M.+Wensing%2C+B.+Katz%2C+G.+Bledt%2C+S.+Kim%2C+Dynamic+locomotion+in+the+MIT+Cheetah+3+through+convex+model-predictive+control%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+And+Systems+%28IROS%29+%28IEEE%2C+2018%29%2C+pp.+1%E2%80%939.)

13

M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, W. Zaremba, Learning dexterous in-hand manipulation. _Int. J. Robot. Res._ **39**, 3–20 (2020).

[Crossref](https://doi.org/10.1177/0278364919887447)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000497268200001)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Learning+dexterous+in-hand+manipulation&author=M.+Andrychowicz&author=B.+Baker&author=M.+Chociej&author=R.+J%C3%B3zefowicz&author=B.+McGrew&author=J.+Pachocki&author=A.+Petron&author=M.+Plappert&author=G.+Powell&author=A.+Ray&author=J.+Schneider&author=S.+Sidor&publication_year=2020&journal=Int.+J.+Robot.+Res.&pages=3-20&doi=10.1177%2F0278364919887447)

14

OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, L. Zhang, Solving Rubik’s cube with a robot hand. arXiv: [1910.07113](https://arxiv.org/abs/1910.07113) (2019).

[Google Scholar](https://scholar.google.com/scholar?q=OpenAI%2C+I.+Akkaya%2C+M.+Andrychowicz%2C+M.+Chociej%2C+M.+Litwin%2C+B.+McGrew%2C+A.+Petron%2C+A.+Paino%2C+M.+Plappert%2C+G.+Powell%2C+R.+Ribas%2C+J.+Schneider%2C+N.+Tezak%2C+J.+Tworek%2C+P.+Welinder%2C+L.+Weng%2C+Q.+Yuan%2C+W.+Zaremba%2C+L.+Zhang%2C+Solving+Rubik%E2%80%99s+cube+with+a+robot+hand.+arXiv%3A1910.07113+%282019%29.)

15

A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, Y. Narang, DeXtreme: Transfer of agile in-hand manipulation from simulation to reality, in _Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA)_ (IEEE, 2023), pp. 5977–5984.

[Google Scholar](https://scholar.google.com/scholar?q=A.+Handa%2C+A.+Allshire%2C+V.+Makoviychuk%2C+A.+Petrenko%2C+R.+Singh%2C+J.+Liu%2C+D.+Makoviichuk%2C+K.+Van+Wyk%2C+A.+Zhurkevich%2C+B.+Sundaralingam%2C+Y.+Narang%2C+DeXtreme%3A+Transfer+of+agile+in-hand+manipulation+from+simulation+to+reality%2C+in+Proceedings+of+the+2023+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29+%28IEEE%2C+2023%29%2C+pp.+5977%E2%80%935984.)

16

J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, M. Hutter, Learning agile and dynamic motor skills for legged robots. _Sci. Robot._ **4**, eaau5872 (2019).

[Crossref](https://doi.org/10.1126/scirobotics.aau5872)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/33137755/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000455967100002)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Learning+agile+and+dynamic+motor+skills+for+legged+robots&author=J.+Hwangbo&author=J.+Lee&author=A.+Dosovitskiy&author=D.+Bellicoso&author=V.+Tsounis&author=V.+Koltun&author=M.+Hutter&publication_year=2019&journal=Sci.+Robot.&pages=eaau5872&doi=10.1126%2Fscirobotics.aau5872&pmid=33137755)

17

J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, M. Hutter, Learning quadrupedal locomotion over challenging terrain. _Sci. Robot._ **5**, eabc5986 (2020).

[Crossref](https://doi.org/10.1126/scirobotics.abc5986)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/33087482/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000581106800002)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Learning+quadrupedal+locomotion+over+challenging+terrain&author=J.+Lee&author=J.+Hwangbo&author=L.+Wellhausen&author=V.+Koltun&author=M.+Hutter&publication_year=2020&journal=Sci.+Robot.&pages=eabc5986&doi=10.1126%2Fscirobotics.abc5986&pmid=33087482)

18

A. Kumar, Z. Fu, D. Pathak, J. Malik, RMA: Rapid motor adaptation for legged robots, _Proceedings of the Robotics: Science and Systems (RSS)_; Virtual Event, 12 to 16 July 2021.

[Google Scholar](https://scholar.google.com/scholar?q=A.+Kumar%2C+Z.+Fu%2C+D.+Pathak%2C+J.+Malik%2C+RMA%3A+Rapid+motor+adaptation+for+legged+robots%2C+Proceedings+of+the+Robotics%3A+Science+and+Systems+%28RSS%29%3B+Virtual+Event%2C+12+to+16+July+2021.)

19

H. Benbrahim, J. A. Franklin, Biped dynamic walking using reinforcement learning. _Rob. Auton. Syst._ **22**, 283–302 (1997).

[Crossref](https://doi.org/10.1016/S0921-8890(97)00043-2)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000071744100007)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Biped+dynamic+walking+using+reinforcement+learning&author=H.+Benbrahim&author=J.+A.+Franklin&publication_year=1997&journal=Rob.+Auton.+Syst.&pages=283-302&doi=10.1016%2FS0921-8890%2897%2900043-2)

20

R. Tedrake, T. W. Zhang, H. S. Seung, Stochastic policy gradient reinforcement learning on a simple 3D biped, in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2004), vol. 3, pp. 2849–2854.

[Google Scholar](https://scholar.google.com/scholar?q=R.+Tedrake%2C+T.+W.+Zhang%2C+H.+S.+Seung%2C+Stochastic+policy+gradient+reinforcement+learning+on+a+simple+3D+biped%2C+in+2004+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2004%29%2C+vol.+3%2C+pp.+2849%E2%80%932854.)

21

Z. Xie, G. Berseth, P. Clary, J. Hurst, M. van de Panne, Feedback control for cassie with deep reinforcement learning, in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2018), pp. 1241–1246.

[Google Scholar](https://scholar.google.com/scholar?q=Z.+Xie%2C+G.+Berseth%2C+P.+Clary%2C+J.+Hurst%2C+M.+van+de+Panne%2C+Feedback+control+for+cassie+with+deep+reinforcement+learning%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2018%29%2C+pp.+1241%E2%80%931246.)

22

J. Siekmann, Y. Godse, A. Fern, J. Hurst, Sim-to-real learning of all common bipedal gaits via periodic reward composition, in _IEEE International Conference on Robotics and Automation (ICRA)_ (IEEE, 2021), pp. 7309–7315.

[Google Scholar](https://scholar.google.com/scholar?q=J.+Siekmann%2C+Y.+Godse%2C+A.+Fern%2C+J.+Hurst%2C+Sim-to-real+learning+of+all+common+bipedal+gaits+via+periodic+reward+composition%2C+in+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29+%28IEEE%2C+2021%29%2C+pp.+7309%E2%80%937315.)

23

J. Siekmann, K. Green, J. Warila, A. Fern, J. Hurst, Blind bipedal stair traversal via sim-to-real reinforcement learning, in _Proceedings of the Robotics: Science and Systems (RSS)_ (RSS, 2021).

[Google Scholar](https://scholar.google.com/scholar?q=J.+Siekmann%2C+K.+Green%2C+J.+Warila%2C+A.+Fern%2C+J.+Hurst%2C+Blind+bipedal+stair+traversal+via+sim-to-real+reinforcement+learning%2C+in+Proceedings+of+the+Robotics%3A+Science+and+Systems+%28RSS%29+%28RSS%2C+2021%29.)

24

S. Iida, S. Kato, K. Kuwayama, T. Kunitachi, M. Kanoh, H. Itoh, Humanoid robot control based on reinforcement learning, in _Micro-Nanomechatronics and Human Science, 2004 and The Fourth Symposium Micro-Nanomechatronics for Information-Based Society, 2004_ (IEEE, 2004), pp. 353–358.

[Google Scholar](https://scholar.google.com/scholar?q=S.+Iida%2C+S.+Kato%2C+K.+Kuwayama%2C+T.+Kunitachi%2C+M.+Kanoh%2C+H.+Itoh%2C+Humanoid+robot+control+based+on+reinforcement+learning%2C+in+Micro-Nanomechatronics+and+Human+Science%2C+2004+and+The+Fourth+Symposium+Micro-Nanomechatronics+for+Information-Based+Society%2C+2004+%28IEEE%2C+2004%29%2C+pp.+353%E2%80%93358.)

25

D. Rodriguez, S. Behnke, Deepwalk: Omnidirectional bipedal gait by deep reinforcement learning, in _2021 IEEE International Conference on Robotics and Automation (ICRA)_ (IEEE, 2021), pp. 3033–3039.

[Google Scholar](https://scholar.google.com/scholar?q=D.+Rodriguez%2C+S.+Behnke%2C+Deepwalk%3A+Omnidirectional+bipedal+gait+by+deep+reinforcement+learning%2C+in+2021+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29+%28IEEE%2C+2021%29%2C+pp.+3033%E2%80%933039.)

26

G. A. Castillo, B. Weng, W. Zhang, A. Hereid, Reinforcement learning-based cascade motion policy design for robust 3D bipedal locomotion. _IEEE Access_ **10**, 20135–20148 (2022).

[Crossref](https://doi.org/10.1109/ACCESS.2022.3151771)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000764079000001)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Reinforcement+learning-based+cascade+motion+policy+design+for+robust+3D+bipedal+locomotion&author=G.+A.+Castillo&author=B.+Weng&author=W.+Zhang&author=A.+Hereid&publication_year=2022&journal=IEEE+Access&pages=20135-20148&doi=10.1109%2FACCESS.2022.3151771)

27

L. Krishna, G. A. Castillo, U. A. Mishra, A. Hereid, S. Kolathaya, Linear policies are sufficient to realize robust bipedal walking on challenging terrains. _IEEE Robot. Autom. Lett._ **7**, 2047–2054 (2022).

[Crossref](https://doi.org/10.1109/LRA.2022.3143227)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000748560800016)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Linear+policies+are+sufficient+to+realize+robust+bipedal+walking+on+challenging+terrains&author=L.+Krishna&author=G.+A.+Castillo&author=U.+A.+Mishra&author=A.+Hereid&author=S.+Kolathaya&publication_year=2022&journal=IEEE+Robot.+Autom.+Lett.&pages=2047-2054&doi=10.1109%2FLRA.2022.3143227)

28

R. Antonova, S. Cruciani, C. Smith, D. Kragic, Reinforcement learning for pivoting task, arXiv: [1703.00472](https://arxiv.org/abs/1703.00472) (2017).

[Google Scholar](https://scholar.google.com/scholar?q=R.+Antonova%2C+S.+Cruciani%2C+C.+Smith%2C+D.+Kragic%2C+Reinforcement+learning+for+pivoting+task%2C+arXiv%3A1703.00472+%282017%29.)

29

F. Sadeghi, S. Levine, Cad2rl: Real single-image flight without a single real image, in _Proceedings of the Robotics: Science and Systems (RSS)_ (RSS, 2016).

[Google Scholar](https://scholar.google.com/scholar?q=F.+Sadeghi%2C+S.+Levine%2C+Cad2rl%3A+Real+single-image+flight+without+a+single+real+image%2C+in+Proceedings+of+the+Robotics%3A+Science+and+Systems+%28RSS%29+%28RSS%2C+2016%29.)

30

J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, P. Abbeel, Domain randomization for transferring deep neural networks from simulation to the real world, in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2017), pp. 23–30.

[Google Scholar](https://scholar.google.com/scholar?q=J.+Tobin%2C+R.+Fong%2C+A.+Ray%2C+J.+Schneider%2C+W.+Zaremba%2C+P.+Abbeel%2C+Domain+randomization+for+transferring+deep+neural+networks+from+simulation+to+the+real+world%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2017%29%2C+pp.+23%E2%80%9330.)

31

X. B. Peng, M. Andrychowicz, W. Zaremba, P. Abbeel, Sim-to-real transfer of robotic control with dynamics randomization, in _IEEE International Conference on Robotics and Automation (ICRA)_ (IEEE, 2018), pp. 3803–3810.

[Google Scholar](https://scholar.google.com/scholar?q=X.+B.+Peng%2C+M.+Andrychowicz%2C+W.+Zaremba%2C+P.+Abbeel%2C+Sim-to-real+transfer+of+robotic+control+with+dynamics+randomization%2C+in+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29+%28IEEE%2C+2018%29%2C+pp.+3803%E2%80%933810.)

32

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. M. Candlish, A. Radford, I. Sutskever, D. Amodei, Language models are few-shot learners, in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ (NeurIPS, 2020), pp. 1877–1901.

[Google Scholar](https://scholar.google.com/scholar?q=T.+Brown%2C+B.+Mann%2C+N.+Ryder%2C+M.+Subbiah%2C+J.+D.+Kaplan%2C+P.+Dhariwal%2C+A.+Neelakantan%2C+P.+Shyam%2C+G.+Sastry%2C+A.+Askell%2C+S.+Agarwal%2C+A.+Herbert-Voss%2C+G.+Krueger%2C+T.+Henighan%2C+R.+Child%2C+A.+Ramesh%2C+D.+M.+Ziegler%2C+J.+Wu%2C+C.+Winter%2C+C.+Hesse%2C+M.+Chen%2C+E.+Sigler%2C+M.+Litwin%2C+S.+Gray%2C+B.+Chess%2C+J.+Clark%2C+C.+Berner%2C+S.+M.+Candlish%2C+A.+Radford%2C+I.+Sutskever%2C+D.+Amodei%2C+Language+models+are+few-shot+learners%2C+in+Proceedings+of+the+34th+International+Conference+on+Neural+Information+Processing+Systems+%28NeurIPS%2C+2020%29%2C+pp.+1877%E2%80%931901.)

33

A. K. Han, A. Hajj-Ahmad, M. R. Cutkosky, Bimanual handling of deformable objects with hybrid adhesion. _IEEE Robot. Autom. Lett._ **7**, 5497–5503 (2022).

[Crossref](https://doi.org/10.1109/LRA.2022.3158231)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000770589900013)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Bimanual+handling+of+deformable+objects+with+hybrid+adhesion&author=A.+K.+Han&author=A.+Hajj-Ahmad&author=M.+R.+Cutkosky&publication_year=2022&journal=IEEE+Robot.+Autom.+Lett.&pages=5497-5503&doi=10.1109%2FLRA.2022.3158231)

34

G. A. Castillo, B. Weng, W. Zhang, A. Hereid, Robust feedback motion policy design using reinforcement learning on a 3D digit bipedal robot, in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2021), pp. 5136–5143.

[Google Scholar](https://scholar.google.com/scholar?q=G.+A.+Castillo%2C+B.+Weng%2C+W.+Zhang%2C+A.+Hereid%2C+Robust+feedback+motion+policy+design+using+reinforcement+learning+on+a+3D+digit+bipedal+robot%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2021%29%2C+pp.+5136%E2%80%935143.)

35

Y. Gao, Y. Gong, V. Paredes, A. Hereid, Y. Gu, Time-varying alip model and robust foot-placement control for underactuated bipedal robotic walking on a swaying rigid surface, in _2023 American Control Conference (ACC)_ (IEEE, 2023), pp. 3282–3287.

[Google Scholar](https://scholar.google.com/scholar?q=Y.+Gao%2C+Y.+Gong%2C+V.+Paredes%2C+A.+Hereid%2C+Y.+Gu%2C+Time-varying+alip+model+and+robust+foot-placement+control+for+underactuated+bipedal+robotic+walking+on+a+swaying+rigid+surface%2C+in+2023+American+Control+Conference+%28ACC%29+%28IEEE%2C+2023%29%2C+pp.+3282%E2%80%933287.)

36

Y. Gao, C. Yuan, Y. Gu, Invariant filtering for legged humanoid locomotion on a dynamic rigid surface. _IEEE ASME Trans. Mechatron._ **27**, 1900–1909 (2022).

[Crossref](https://doi.org/10.1109/TMECH.2022.3176015)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000836682800001)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Invariant+filtering+for+legged+humanoid+locomotion+on+a+dynamic+rigid+surface&author=Y.+Gao&author=C.+Yuan&author=Y.+Gu&publication_year=2022&journal=IEEE+ASME+Trans.+Mechatron.&pages=1900-1909&doi=10.1109%2FTMECH.2022.3176015)

37

A. Adu-Bredu, N. Devraj, O. C. Jenkins, Optimal constrained task planning as mixed integer programming, in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ (IEEE, 2022), pp. 12029–12036.

[Google Scholar](https://scholar.google.com/scholar?q=A.+Adu-Bredu%2C+N.+Devraj%2C+O.+C.+Jenkins%2C+Optimal+constrained+task+planning+as+mixed+integer+programming%2C+in+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29+%28IEEE%2C+2022%29%2C+pp.+12029%E2%80%9312036.)

38

K. S. Narkhede, A. M. Kulkarni, D. A. Thanki, I. Poulakakis, A sequential mpc approach to reactive planning for bipedal robots using safe corridors in highly cluttered environments. _IEEE Robot. Autom. Lett._ **7**, 11831–11838 (2022).

[Crossref](https://doi.org/10.1109/LRA.2022.3204367)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000861433400002)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=A+sequential+mpc+approach+to+reactive+planning+for+bipedal+robots+using+safe+corridors+in+highly+cluttered+environments&author=K.+S.+Narkhede&author=A.+M.+Kulkarni&author=D.+A.+Thanki&author=I.+Poulakakis&publication_year=2022&journal=IEEE+Robot.+Autom.+Lett.&pages=11831-11838&doi=10.1109%2FLRA.2022.3204367)

39

A. Shamsah, Z. Gu, J. Warnke, S. Hutchinson, Y. Zhao, Integrated task and motion planning for safe legged navigation in partially observable environments. _IEEE Trans. Robot._ **39**, 4913–4934 (2023).

[Crossref](https://doi.org/10.1109/TRO.2023.3299524)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=001051270800001)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Integrated+task+and+motion+planning+for+safe+legged+navigation+in+partially+observable+environments&author=A.+Shamsah&author=Z.+Gu&author=J.+Warnke&author=S.+Hutchinson&author=Y.+Zhao&publication_year=2023&journal=IEEE+Trans.+Robot.&pages=4913-4934&doi=10.1109%2FTRO.2023.3299524)

40

D. J. Morton, D. D. Fuller, _Human Locomotion and Body Form: A Study of Gravity and Man_ (Williams & Wilkins, 1952).

[Google Scholar](https://scholar.google.com/scholar?q=D.+J.+Morton%2C+D.+D.+Fuller%2C+Human+Locomotion+and+Body+Form%3A+A+Study+of+Gravity+and+Man+%28Williams+%26+Wilkins%2C+1952%29.)

41

H. Herr, M. Popovic, Angular momentum in human walking. _J. Exp. Biol._ **211**, 467–481 (2008).

[Crossref](https://doi.org/10.1242/jeb.008573)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/18245623/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000253196500005)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Angular+momentum+in+human+walking&author=H.+Herr&author=M.+Popovic&publication_year=2008&journal=J.+Exp.+Biol.&pages=467-481&doi=10.1242%2Fjeb.008573&pmid=18245623)

42

S. H. Collins, P. G. Adamczyk, A. D. Kuo, Dynamic arm swinging in human walking. _Proc. R. Soc. B Biol. Sci._ **276**, 3679–3688 (2009).

[Crossref](https://doi.org/10.1098/rspb.2009.0664)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/19640879/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000270172200014)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Dynamic+arm+swinging+in+human+walking&author=S.+H.+Collins&author=P.+G.+Adamczyk&author=A.+D.+Kuo&publication_year=2009&journal=Proc.+R.+Soc.+B+Biol.+Sci.&pages=3679-3688&doi=10.1098%2Frspb.2009.0664&pmid=19640879)

43

J. D. Ortega, L. A. Fehlman, C. T. Farley, Effects of aging and arm swing on the metabolic cost of stability in human walking. _J. Biomech._ **41**, 3303–3308 (2008).

[Crossref](https://doi.org/10.1016/j.jbiomech.2008.06.039)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/18814873/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000261965300002)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Effects+of+aging+and+arm+swing+on+the+metabolic+cost+of+stability+in+human+walking&author=J.+D.+Ortega&author=L.+A.+Fehlman&author=C.+T.+Farley&publication_year=2008&journal=J.+Biomech.&pages=3303-3308&doi=10.1016%2Fj.jbiomech.2008.06.039&pmid=18814873)

44

B. R. Umberger, Effects of suppressing arm swing on kinematics, kinetics, and energetics of human walking. _J. Biomech._ **41**, 2575–2580 (2008).

[Crossref](https://doi.org/10.1016/j.jbiomech.2008.05.024)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/18621376/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000259129000031)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Effects+of+suppressing+arm+swing+on+kinematics%2C+kinetics%2C+and+energetics+of+human+walking&author=B.+R.+Umberger&publication_year=2008&journal=J.+Biomech.&pages=2575-2580&doi=10.1016%2Fj.jbiomech.2008.05.024&pmid=18621376)

45

M. Murray, S. Sepic, E. Barnard, Patterns of sagittal rotation of the upper limbs in walking. _Phys. Ther._ **47**, 272–284 (1967).

[Crossref](https://doi.org/10.1093/ptj/47.4.272)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/6040248/)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Patterns+of+sagittal+rotation+of+the+upper+limbs+in+walking&author=M.+Murray&author=S.+Sepic&author=E.+Barnard&publication_year=1967&journal=Phys.+Ther.&pages=272-284&doi=10.1093%2Fptj%2F47.4.272&pmid=6040248)

46

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J.Wu, D. Amodei, Scaling laws for neural language models. arXiv: [2001.08361](https://arxiv.org/abs/2001.08361) (2020).

[Google Scholar](https://scholar.google.com/scholar?q=J.+Kaplan%2C+S.+McCandlish%2C+T.+Henighan%2C+T.+B.+Brown%2C+B.+Chess%2C+R.+Child%2C+S.+Gray%2C+A.+Radford%2C+J.Wu%2C+D.+Amodei%2C+Scaling+laws+for+neural+language+models.+arXiv%3A2001.08361+%282020%29.)

47

J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. L. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Bińkowski, R. Barreira, O. Vinyals, A. Zisserman, K. Simonyan, Flamingo: A visual language model for few-shot learning. _Adv. Neural Inf. Process. Syst._ **35**, 23716 (2022).

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Flamingo%3A+A+visual+language+model+for+few-shot+learning&author=J.-B.+Alayrac&author=J.+Donahue&author=P.+Luc&author=A.+Miech&author=I.+Barr&author=Y.+Hasson&author=K.+Lenc&author=A.+Mensch&author=K.+Millican&author=M.+Reynolds&author=R.+Ring&author=E.+Rutherford&publication_year=2022&journal=Adv.+Neural+Inf.+Process.+Syst.&pages=23716)

48

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale, in _International Conference on Learning Representations_ (2021).

[Google Scholar](https://scholar.google.com/scholar?q=A.+Dosovitskiy%2C+L.+Beyer%2C+A.+Kolesnikov%2C+D.+Weissenborn%2C+X.+Zhai%2C+T.+Unterthiner%2C+M.+Dehghani%2C+M.+Minderer%2C+G.+Heigold%2C+S.+Gelly%2C+J.+Uszkoreit%2C+N.+Houlsby%2C+An+image+is+worth+16x16+words%3A+Transformers+for+image+recognition+at+scale%2C+in+International+Conference+on+Learning+Representations+%282021%29.)

49

A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, Improving language understanding by generative pre-training (2018).

[Google Scholar](https://scholar.google.com/scholar?q=A.+Radford%2C+K.+Narasimhan%2C+T.+Salimans%2C+I.+Sutskever%2C+Improving+language+understanding+by+generative+pre-training+%282018%29.)

50

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, _Proceedings of the 31st International Conference on Neural Information Processing Systems_ (NeurIPS, 2017), pp. 6000–6010.

[Google Scholar](https://scholar.google.com/scholar?q=A.+Vaswani%2C+N.+Shazeer%2C+N.+Parmar%2C+J.+Uszkoreit%2C+L.+Jones%2C+A.+N.+Gomez%2C+%C5%81.+Kaiser%2C+I.+Polosukhin%2C+Attention+is+all+you+need%2C+Proceedings+of+the+31st+International+Conference+on+Neural+Information+Processing+Systems+%28NeurIPS%2C+2017%29%2C+pp.+6000%E2%80%936010.)

51

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, arXiv: [1810.04805](https://arxiv.org/abs/1810.04805) (2018).

[Google Scholar](https://scholar.google.com/scholar?q=J.+Devlin%2C+M.-W.+Chang%2C+K.+Lee%2C+K.+Toutanova%2C+BERT%3A+Pre-training+of+deep+bidirectional+transformers+for+language+understanding%2C+arXiv%3A1810.04805+%282018%29.)

52

L. Dong, S. Xu, B. Xu, Speech-transformer: A no-recurrence sequence-to-sequence model for speech recognition, in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ (IEEE, 2018), pp. 5884–5888.

[Google Scholar](https://scholar.google.com/scholar?q=L.+Dong%2C+S.+Xu%2C+B.+Xu%2C+Speech-transformer%3A+A+no-recurrence+sequence-to-sequence+model+for+speech+recognition%2C+in+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29+%28IEEE%2C+2018%29%2C+pp.+5884%E2%80%935888.)

53

N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, S. Zagoruyko, End-to-end object detection with transformers, in _ECCV 2020: 16th European Conference_, vol. 12346 of _Lecture Notes in Computer Science_ (Springer, 2020), pp. 213–229.

[Google Scholar](https://scholar.google.com/scholar?q=N.+Carion%2C+F.+Massa%2C+G.+Synnaeve%2C+N.+Usunier%2C+A.+Kirillov%2C+S.+Zagoruyko%2C+End-to-end+object+detection+with+transformers%2C+in+ECCV+2020%3A+16th+European+Conference%2C+vol.+12346+of+Lecture+Notes+in+Computer+Science+%28Springer%2C+2020%29%2C+pp.+213%E2%80%93229.)

54

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, arXiv: [1707.06347](https://arxiv.org/abs/1707.06347) (2017).

[Google Scholar](https://scholar.google.com/scholar?q=J.+Schulman%2C+F.+Wolski%2C+P.+Dhariwal%2C+A.+Radford%2C+O.+Klimov%2C+Proximal+policy+optimization+algorithms%2C+arXiv%3A1707.06347+%282017%29.)

55

X. Da, O. Harib, R. Hartley, B. Griffin, J. W. Grizzle, From 2d design of underactuated bipedal gaits to 3d implementation: Walking with speed tracking. _IEEE Access_ **4**, 3469–3478 (2016).

[Crossref](https://doi.org/10.1109/ACCESS.2016.2582731)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=000381468600003)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=From+2d+design+of+underactuated+bipedal+gaits+to+3d+implementation%3A+Walking+with+speed+tracking&author=X.+Da&author=O.+Harib&author=R.+Hartley&author=B.+Griffin&author=J.+W.+Grizzle&publication_year=2016&journal=IEEE+Access&pages=3469-3478&doi=10.1109%2FACCESS.2016.2582731)

56

Y. Gong, R. Hartley, X. Da, A. Hereid, O. Harib, J.-K. Huang, J. Grizzle, Feedback control of a cassie bipedal robot: Walking, standing, and riding a Segway, in _2019 American Control Conference (ACC)_ (IEEE, 2019), pp. 4559–4566.

[Google Scholar](https://scholar.google.com/scholar?q=Y.+Gong%2C+R.+Hartley%2C+X.+Da%2C+A.+Hereid%2C+O.+Harib%2C+J.-K.+Huang%2C+J.+Grizzle%2C+Feedback+control+of+a+cassie+bipedal+robot%3A+Walking%2C+standing%2C+and+riding+a+Segway%2C+in+2019+American+Control+Conference+%28ACC%29+%28IEEE%2C+2019%29%2C+pp.+4559%E2%80%934566.)

57

V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, G. State, Isaac Gym: High performance GPU-based physics simulation for robot learning, arXiv: [2108.10470](https://arxiv.org/abs/2108.10470) (2021).

[Google Scholar](https://scholar.google.com/scholar?q=V.+Makoviychuk%2C+L.+Wawrzyniak%2C+Y.+Guo%2C+M.+Lu%2C+K.+Storey%2C+M.+Macklin%2C+D.+Hoeller%2C+N.+Rudin%2C+A.+Allshire%2C+A.+Handa%2C+G.+State%2C+Isaac+Gym%3A+High+performance+GPU-based+physics+simulation+for+robot+learning%2C+arXiv%3A2108.10470+%282021%29.)

58

N. Rudin, D. Hoeller, P. Reist, M. Hutter, Learning to walk in minutes using massively parallel deep reinforcement learning, in _Conference on Robot Learning_ (MLResearchPress, 2022).

[Google Scholar](https://scholar.google.com/scholar?q=N.+Rudin%2C+D.+Hoeller%2C+P.+Reist%2C+M.+Hutter%2C+Learning+to+walk+in+minutes+using+massively+parallel+deep+reinforcement+learning%2C+in+Conference+on+Robot+Learning+%28MLResearchPress%2C+2022%29.)

59

S. Bai, J. Z. Kolter, V. Koltun, An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv: [1803.01271](https://arxiv.org/abs/1803.01271) (2018).

[Google Scholar](https://scholar.google.com/scholar?q=S.+Bai%2C+J.+Z.+Kolter%2C+V.+Koltun%2C+An+empirical+evaluation+of+generic+convolutional+and+recurrent+networks+for+sequence+modeling%2C+arXiv%3A1803.01271+%282018%29.)

60

S. Hochreiter, J. Schmidhuber, Long short-term memory. _Neural Comput._ **9**, 1735–1780 (1997).

[Crossref](https://doi.org/10.1162/neco.1997.9.8.1735)

[PubMed](https://pubmed.ncbi.nlm.nih.gov/9377276/)

[Web of Science](https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&DestApp=WOS_CPL&UsrCustomerID=5e3815c904498985e796fc91436abd9a&SrcAuth=atyponcel&SrcApp=literatum&DestLinkType=FullRecord&KeyUT=A1997YA04500007)

[Google Scholar](https://scholar.google.com/scholar_lookup?title=Long+short-term+memory&author=S.+Hochreiter&author=J.+Schmidhuber&publication_year=1997&journal=Neural+Comput.&pages=1735-1780&doi=10.1162%2Fneco.1997.9.8.1735&pmid=9377276)

61

J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke, Sim-to-real: Learning agile locomotion for quadruped robots, in _Proceedings of the Robotics: Science and Systems (RSS)_ (RSS, 2018).

[Google Scholar](https://scholar.google.com/scholar?q=J.+Tan%2C+T.+Zhang%2C+E.+Coumans%2C+A.+Iscen%2C+Y.+Bai%2C+D.+Hafner%2C+S.+Bohez%2C+and+V.+Vanhoucke%2C+Sim-to-real%3A+Learning+agile+locomotion+for+quadruped+robots%2C+in+Proceedings+of+the+Robotics%3A+Science+and+Systems+%28RSS%29+%28RSS%2C+2018%29.)

### Submit a Response to This Article

×

##### Compose eLetter

Title:

Title is required

Contents:

##### Contributors

remove contributor

First name:

Last name:

Email:

Role/occupation:

affiliation:

add another contributor

##### Statement of Competing Interests

Competing interests?

YES

NO

Please describe the competing interests

CANCELSUBMIT

## (0)eLetters

eLetters is a forum for ongoing peer review. eLetters are not edited, proofread, or indexed, but they are screened. eLetters should provide substantive and scholarly commentary on the article. Embedded figures cannot be submitted, and we discourage the use of figures within eLetters in general. If a figure is essential, please include a link to the figure within the text of the eLetter. Please read our [Terms of Service](/content/page/terms-service) before submitting an eLetter.

[Log In to Submit a Response](/action/ssostart?redirectUri=/doi/full/10.1126/scirobotics.adi9579)

No eLetters have been published for this article yet.

## Information & Authors

### Information

#### Published In

![](/cms/asset/5ea75a1e-d622-4687-a40b-e4ed742a5e49/scirobotics.2024.9.issue-89.largecover.jpg)

Science Robotics

Volume 9 \| Issue 89

April 2024

#### Copyright

Copyright © 2024 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works.

[https://www.sciencemag.org/about/science-licenses-journal-article-reuse](https://www.sciencemag.org/about/science-licenses-journal-article-reuse)

This is an article distributed under the terms of the [Science Journals Default License](https://www.science.org/content/page/science-licenses-journal-article-reuse).

#### Article versions

#### Submission history

**Received**: 31 May 2023

**Accepted**: 26 March 2024

#### Permissions

Request permissions for this article.

[Request Permissions](https://s100.copyright.com/AppDispatchServlet?publisherName=AAAS&publication=robotics&title=Real-world+humanoid+locomotion+with+reinforcement+learning&publicationDate=2024-04-17&author=Ilija+Radosavovic%2C+Tete+Xiao%2C+Bike+Zhang%2C+Trevor+Darrell%2C+et+al.&contentID=10.1126%2Fscirobotics.adi9579&volumeNum=9&issueNum=89&orderBeanReset=true)

#### Acknowledgments

We thank S. Kamat, B. Shi, and S. Cakir for help with experiments; Y. Liu for discussions of spring approximation in simulation; A. Srinivas, A. Gupta, A. Kumar, W. Peebles, T. Brooks, M. Tancik, S. Chen, Z. Li, B. McInroe, and R. Huang for helpful discussions; G. State, P. Reist, V. Makoviychuk, A. Handa, and the I. Gym team for simulation discussions; J. Thomas, J. Thompson, L. Allery, J. Hurst, and the Agility Robotics team for hardware discussions.

**Funding:** This work was supported in part by DARPA Machine Common Sense program, ONR MURI program (N00014-21-1-2801), NVIDIA, InnoHK of the Government of the Hong Kong Special Administrative Region via the Hong Kong Centre for Logistics Robotics, the AI Institute, and BAIR’s industrial alliance programs.

**Author contributions:** I.R., T.X., and B.Z. led the project. T.D., J.M., and K.S. advised the project.

**Competing interests:** J.M. is also (part-time) affiliated with Meta Platforms Inc., but this research was not sponsored or supported by Meta. The other authors declare that they have no competing interests.

**Data and materials availability:** Additional data and materials are present in the Supplementary Materials.

### Authors

#### Affiliations

IlijaRadosavovic[\*](#cor1), [†](#afn1) [https://orcid.org/0009-0004-9200-5980](https://orcid.org/0009-0004-9200-5980) [\[email protected\]](/cdn-cgi/l/email-protection#98f1f4f1f2f9d8fafdeaf3fdf4fde1b6fdfceda7fbfba5ece0f1f9f7d8fafdeaf3fdf4fde1b6fdfcedbef9f5e8a3fbfba5faf1f3fde2f0f9f6ffd8fafdeaf3fdf4fde1b6fdfced)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Supervision, Validation, Visualization, Writing - original draft, and Writing - review & editing.

[View all articles by this author](/authored-by/Radosavovic/Ilija)

TeteXiao[\*](#cor1), [†](#afn1) [https://orcid.org/0009-0006-6440-2393](https://orcid.org/0009-0006-6440-2393) [\[email protected\]](/cdn-cgi/l/email-protection#41282d282b20012324332a242d24386f2425347e22227c353928202e012324332a242d24386f24253467202c317a22227c23282a243b29202f26012324332a242d24386f242534)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, and Writing - review & editing.

[View all articles by this author](/authored-by/Xiao/Tete)

BikeZhang[\*](#cor1), [†](#afn1) [https://orcid.org/0000-0002-2652-4318](https://orcid.org/0000-0002-2652-4318) [\[email protected\]](/cdn-cgi/l/email-protection#b1d8ddd8dbd0f1d3d4c3dad4ddd4c89fd4d5c48ed2d28cc5c9d8d0def1d3d4c3dad4ddd4c89fd4d5c497d0dcc18ad2d28cd3d8dad4cbd9d0dfd6f1d3d4c3dad4ddd4c89fd4d5c4)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, and Writing - review & editing.

[View all articles by this author](/authored-by/Zhang/Bike)

TrevorDarrell[‡](#afn2)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Visualization, and Writing - review & editing.

[View all articles by this author](/authored-by/Darrell/Trevor)

JitendraMalik[‡](#afn2) [https://orcid.org/0000-0003-3695-1580](https://orcid.org/0000-0003-3695-1580)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing - original draft, and Writing - review & editing.

[View all articles by this author](/authored-by/Malik/Jitendra)

KoushilSreenath[‡](#afn2) [https://orcid.org/0000-0002-5346-3637](https://orcid.org/0000-0002-5346-3637)

University of California, Berkeley CA, USA.

Roles: Conceptualization, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Visualization, and Writing - review & editing.

[View all articles by this author](/authored-by/Sreenath/Koushil)

#### Funding Information

[Office of Naval Research](http://dx.doi.org/10.13039/100000006): N00014-21-1-2801

[Defense Sciences Office, DARPA](http://dx.doi.org/10.13039/100006502): Machine Common Sense

#### Notes

\*

Corresponding author. Email: [\[email protected\]](/cdn-cgi/l/email-protection#71181d181b10311314031a141d14085f141504) (I.R.); [\[email protected\]](/cdn-cgi/l/email-protection#20545849414f604245524b454c45590e454455) (T.X.); [\[email protected\]](/cdn-cgi/l/email-protection#284a41434d524049464f684a4d5a434d444d51064d4c5d) (B.Z.)

†

These authors contributed equally to this work.

‡

These authors contributed equally to this work.

## Metrics & Citations

### Metrics

#### Article Usage

**Note:** The article usage is presented with a three- to four-day delay and will update daily once available. Due to this delay, usage data will not appear immediately following publication.

Citation information is sourced from [Crossref Cited-by](https://www.crossref.org/services/cited-by/ "Follow link") service.

#### Altmetrics

### Citations

#### Cite as

- Ilija Radosavovic _et al._

,

Real-world humanoid locomotion with reinforcement learning._Sci. Robot._**9**,eadi9579(2024).DOI: [10.1126/scirobotics.adi9579](https://doi.org/10.1126/scirobotics.adi9579)

#### Export citation

Select the format you want to export the citation of this publication.

Please select one from the listRIS (ProCite, Reference Manager)EndNoteBibTexMedlarsRefWorks

Direct import


Citation information is sourced from [Crossref Cited-by](https://www.crossref.org/services/cited-by/ "Follow link") service.

## View Options

### Check Access

#### Log in to view the full text

[AAAS ID LOGIN](/action/ssostart?redirectUri=)

**Loading institution options**

[Select another institution](/action/ssostart?redirectUri=)

AAAS login provides access to Science for AAAS Members, and access to other journals in the Science family to users who have purchased individual subscriptions.

- [Become a AAAS Member](http://www.aaas.org/join/?CTC=SMAAJN)
- [Activate your AAAS ID](https://account.aaas.org/Identity/Lookup)
- [Purchase Access to Other Journals in the Science Family](/content/page/purchase-journal-access)
- [Account Help](/content/page/access-and-subscriptions)

#### More options

[Purchase access to this article](/action/addToCart?offer=ppv&format=ONLINE&doi=10.1126/scirobotics.adi9579)

Download and print this article within 24 hours for your personal scholarly, research, and educational
use.

### View options

#### PDF format

Download this article as a PDF file

[Download PDF](/doi/pdf/10.1126/scirobotics.adi9579?download=true)

#### Full Text

[FULL TEXT](https://www.science.org/doi/full/10.1126/scirobotics.adi9579 "Full Text")

## Media

### Figures

### Multimedia

## Tables

## Share

### Share

#### Share article link

Copy Link

Copied!

Copying failed.

#### Share on social media

FacebookXLinkedinRedditWeChatWhatsAppemail

#### Current Issue

[![Science Robotics cover image](/cdn-cgi/image/width=400/cms/asset/b1025aef-29ed-45c7-a599-600029ee9d23/scirobotics.2024.9.issue-94.largecover.jpg)](/toc/scirobotics/9/94 "View Science Robotics current issue")

### [Stretchable Arduinos embedded in soft robots](/doi/10.1126/scirobotics.adn6844 "Stretchable Arduinos embedded in soft robots")

- By
  - Stephanie J. Woodman
  - Dylan S. Shah
  - _et al._

### [Restoration of grasping in an upper limb amputee using the myokinetic prosthesis with implanted magnets](/doi/10.1126/scirobotics.adp3260 "Restoration of grasping in an upper limb amputee using the myokinetic prosthesis with implanted magnets")

- By
  - Marta Gherardini
  - Valerio Ianniciello
  - _et al._

### [Visual seafloor mapping with autonomous robots](/doi/10.1126/scirobotics.ads9444 "Visual seafloor mapping with autonomous robots")

- By
  - Amos Matsiko

[Table of Contents](/toc/scirobotics/current)

#### Advertisement

### Sign up for ScienceAdviser

Subscribe to ScienceAdviser to get the latest news, commentary, and research, free to your inbox daily.

[Subscribe](/content/page/scienceadviser?intcmp=rrail-adviser&utm_id=recFUzjFNRznSEEDd)

#### LATEST NEWS

[Careers](/news/careers-editorial)12 Sep 2024

[International postdocs in the U.S. are short-changed—in more ways than one](/content/article/international-postdocs-u-s-are-short-changed-more-ways-one "International postdocs in the U.S. are short-changed—in more ways than one")

[News](/news/all-news)12 Sep 2024

[AI chatbot shows promise in talking people out of conspiracy theories](/content/article/ai-chatbot-shows-promise-talking-people-out-conspiracy-theories "AI chatbot shows promise in talking people out of conspiracy theories")

[ScienceInsider](/news/scienceinsider)12 Sep 2024

[Arctic ecosystems get long-term look with drifting research station](/content/article/arctic-ecosystems-get-long-term-look-drifting-research-station "Arctic ecosystems get long-term look with drifting research station")

[ScienceShots](/news/scienceshots)12 Sep 2024

[U.S. state composting laws are mostly trash—except for one, study finds](/content/article/u-s-state-composting-laws-are-mostly-trash-except-one-study-finds "U.S. state composting laws are mostly trash—except for one, study finds")

[News](/news/all-news)12 Sep. 2024

[Strong El Niños primed Earth for mass extinction](/content/article/strong-el-niños-primed-earth-mass-extinction "Strong El Niños primed Earth for mass extinction")

[News](/news/all-news)12 Sep 2024

[News at a glance: Satellite tracks, jailed Russian physicist, and North Korea’s foreign professors](/content/article/news-glance-satellite-tracks-jailed-russian-physicist-and-north-korea-s-foreign "News at a glance: Satellite tracks, jailed Russian physicist, and North Korea’s foreign professors")

#### Advertisement

#### Advertisement

[View full text](/doi/full/10.1126/scirobotics.adi9579) \| [Download PDF](/doi/pdf/10.1126/scirobotics.adi9579)

Now Reading:

SHARE

[**PREVIOUS ARTICLE** \
\
Learning robust autonomous navigation and locomotion for wheeled-legged robots\
\
Previous](/doi/10.1126/scirobotics.adi9641) [**NEXT ARTICLE** \
\
Locomotion as manipulation with ReachBot\
\
Next](/doi/10.1126/scirobotics.adi9762)

Open in viewer

GO TO

GO TO

Show all references

Request permissionsExpand All

Collapse

Expand for more

[Authors Info & Affiliations](#tab-contributors)

SHOW ALL BOOKS

[Skip slideshow](#afterSlideshow-j3n)

[Science Journal Logo](/journal/science "Science Journal Logo")

[Science Advances Journal Logo](/journal/sciadv "Science Advances Journal Logo")

[Science Immunology Journal Logo](/journal/sciimmunol "Science Immunology Journal Logo")

[Science Robotics Journal Logo](/journal/scirobotics "Science Robotics Journal Logo")

[Science Signaling Journal Logo](/journal/signaling "Science Signaling Journal Logo")

[Science Translational Medicine Journal Logo](/journal/stm "Science Translational Medicine Journal Logo")

###### [Follow Us](https://www.science.org/content/page/science-family-journals-social-media)

- [Facebook](https://www.facebook.com/ScienceMagazine "Facebook")
- [Share on X](https://twitter.com/sciencemagazine "Share on X")
- [Instagram](https://www.instagram.com/ScienceMagazine "Instagram")
- [Youtube](https://www.youtube.com/user/ScienceMag "Youtube")
- [RSS feeds](/about/email-alerts-and-rss-feeds "RSS feeds")
- [WeChat](https://mp.weixin.qq.com/s?__biz=MzI3NDY3NzQ2Mg==&mid=100002815&idx=1&sn=2949c025a553ac718b9612a0473b9f60&chksm=6b1120465c66a9508b01eaef1589b15d440e50b189106c8c594de8c6471f696a978de952fb15&mpshare=1&scene=1&srcid=0716JJQ5V4cKbgMMsya2MQ0n&sharer_sharetime= "WeChat")
- [Get our newsletter](/content/page/scienceadviser?intcmp=ftr-adviser&utm_id=recdExfxt1yeSJxzi)

- [NEWS](/news)
- [All News](/news/all-news)
- [ScienceInsider](/news/scienceinsider)
- [News Features](/news/features)
- [Subscribe to News from Science](/content/page/news-science-subscriptions)
- [News from Science FAQ](/content/page/news-subscriber-faqs)
- [About News from Science](/content/page/about-news-science)
- [Donate to News](/news/donate?intcmp=footer-donate&utm_id=recVQVKKRRLdidlGT)

- [CAREERS](/careers)
- [Careers Articles](/topic/article-type/careers-editorial)
- [Find Jobs](https://jobs.sciencecareers.org/)
- [Employer Hubs](/careers/employers)

- [COMMENTARY](/commentary)
- [Opinion](/commentary/opinion)
- [Analysis](/commentary/analysis)
- [Blogs](/blogs)

- [JOURNALS](/journals)
- [Science](/journal/science)
- [Science Advances](/journal/sciadv)
- [Science Immunology](/journal/sciimmunol)
- [Science Robotics](/journal/scirobotics)
- [Science Signaling](/journal/signaling)
- [Science Translational Medicine](/journal/stm)
- [Science Partner Journals](https://spj.sciencemag.org/)

- [AUTHORS & REVIEWERS](/content/page/contributing-science-family-journals)
- [Information for Authors](/content/page/contributing-science-family-journals)
- [Information for Reviewers](/content/page/peer-review-science-publications)

- [LIBRARIANS](/content/page/librarian-portal)
- [Manage Your Institutional Subscription](/action/institutionAccessEntitlements)
- [Library Admin Portal](/content/page/librarian-portal)
- [Request a Quote](https://scienceaaas.org/request)
- [Librarian FAQs](/content/page/librarian-portal-frequently-asked-questions)

- [ADVERTISERS](https://advertising.sciencemag.org/)
- [Advertising Kits](https://advertising.sciencemag.org/)
- [Custom Publishing Info](/custom-publishing)
- [Post a Job](https://employers.sciencecareers.org/)

- [RELATED SITES](/content/page/related-sites)
- [AAAS.org](//www.aaas.org/)
- [AAAS Communities](//members.aaas.org/home)
- [EurekAlert!](//www.eurekalert.org/)
- [Science in the Classroom](//www.scienceintheclassroom.org/)

- [ABOUT US](/content/page/aboutus)
- [Leadership](/content/page/leadership-and-management)
- [Work at AAAS](//www.aaas.org/careers/workataaas)
- [Prizes and Awards](/content/page/prizes-and-awards)

- [HELP](/content/page/help)
- [FAQs](/content/page/faqs)
- [Access and Subscriptions](/content/page/access-and-subscriptions)
- [Order a Single Issue](//backissues.science.org/)
- [Reprints and Permissions](/content/page/reprints-and-permissions)
- [TOC Alerts and RSS Feeds](/content/page/email-alerts-and-rss-feeds)
- [Contact Us](/content/page/contact-us)

###### [Follow Us](https://www.science.org/content/page/science-family-journals-social-media)

- [Facebook](https://www.facebook.com/ScienceMagazine "Facebook")
- [Share on X](https://twitter.com/sciencemagazine "Share on X")
- [Instagram](https://www.instagram.com/ScienceMagazine "Instagram")
- [YouTube](https://www.youtube.com/user/ScienceMag "YouTube")
- [RSS feeds](/content/page/email-alerts-and-rss-feeds "RSS feeds")
- [WeChat](https://mp.weixin.qq.com/s?__biz=MzI3NDY3NzQ2Mg==&mid=100002815&idx=1&sn=2949c025a553ac718b9612a0473b9f60&chksm=6b1120465c66a9508b01eaef1589b15d440e50b189106c8c594de8c6471f696a978de952fb15&mpshare=1&scene=1&srcid=0716JJQ5V4cKbgMMsya2MQ0n&sharer_sharetime= "WeChat")

[Get our newsletter](/content/page/scienceadviser?intcmp=ftr-adviser&utm_id=recdExfxt1yeSJxzi)

[![AAAS logo](/pb-assets/images/styleguide/aaas-logo-1672180581667.svg)](//www.aaas.org/ "Visit the AAAS homepage")

© 2024 American Association for the Advancement of Science. All rights reserved. AAAS is a partner of HINARI, AGORA, OARE, CHORUS, CLOCKSS, CrossRef and COUNTER. _Science Robotics_ eISSN 2470-9476.

back to top

- [Terms of Service](/content/page/terms-service "Terms of Service")
- [Privacy Policy](/content/page/privacy-policy "Privacy Policy")
- [Accessibility](/content/page/accessibility "Accessibility")

[![ScienceAdviser](/pb-assets/images/logos/ScienceAdviser-logo-1683228767297.svg)](/scienceadviser)

Get the latest news, commentary, and research, free to your inbox daily.

[Subscribe](/content/page/scienceadviser)Not Now

×

Back to article