Alexa, play with robot: Introducing the First Alexa Prize SimBot
  Challenge on Embodied AI
The Alexa Prize program has empowered numerous university students to
explore, experiment, and showcase their talents in building conversational
agents through challenges like the SocialBot Grand Challenge and the TaskBot
Challenge. As conversational agents increasingly appear in multimodal and
embodied contexts, it is important to explore the affordances of conversational
interaction augmented with computer vision and physical embodiment. This paper
describes the SimBot Challenge, a new challenge in which university teams
compete to build robot assistants that complete tasks in a simulated physical
environment. This paper provides an overview of the SimBot Challenge, which
included both online and offline challenge phases. We describe the
infrastructure and support provided to the teams including Alexa Arena, the
simulated environment, and the ML toolkit provided to teams to accelerate their
building of vision and language models. We summarize the approaches the
participating teams took to overcome research challenges and extract key
lessons learned. Finally, we provide analysis of the performance of the
competing SimBots during the competition.
64d596d0d9d00dcb114e7472
Hangjie Shi
64d596d0d9d00dcb114e7473
Leslie Ball
64d596d0d9d00dcb114e7474
Govind Thattai
64d596d0d9d00dcb114e7475
Desheng Zhang
64d596d0d9d00dcb114e7476
Lucy Hu
64d596d0d9d00dcb114e7477
Qiaozi Gao
64d596d0d9d00dcb114e7478
Suhaila Shakiah
admin_assigned
2023-08-11T13:31:22.507Z
645d7db5ce72244df7b3d939
/avatars/9fd48ebd3233bdf336664600deff3ea5.svg
Suhaila Shakiah
user
suhailashakiah
64d596d0d9d00dcb114e7479
Xiaofeng Gao
64d596d0d9d00dcb114e747a
Aishwarya Padmakumar
64d596d0d9d00dcb114e747b
Bofei Yang
64d596d0d9d00dcb114e747c
Cadence Chung
64d596d0d9d00dcb114e747d
Dinakar Guthy
64d596d0d9d00dcb114e747e
Gaurav Sukhatme
64d596d0d9d00dcb114e747f
Karthika Arumugam
64d596d0d9d00dcb114e7480
Matthew Wen
64d596d0d9d00dcb114e7481
Osman Ipek
64d596d0d9d00dcb114e7482
Patrick Lange
admin_assigned
2023-08-11T13:34:08.473Z
640f53dd208821a59b740bae
/avatars/460de2ab74dddfbc1502ff87acae314b.svg
Patrick Lange
user
langep
64d596d0d9d00dcb114e7483
Rohan Khanna
64d596d0d9d00dcb114e7484
Shreyas Pansare
64d596d0d9d00dcb114e7485
Vasu Sharma
claimed_verified
2025-01-10T08:22:24.768Z
65492730a3d1682f79f3ab7a
/avatars/fc62605bdf10947dab393305c51ace96.svg
Vasu Sharma
user
vasusharma55
64d596d0d9d00dcb114e7486
Chao Zhang
64d596d0d9d00dcb114e7487
Cris Flagg
64d596d0d9d00dcb114e7488
Daniel Pressel
64d596d0d9d00dcb114e7489
Lavina Vaz
64d596d0d9d00dcb114e748a
Luke Dai
64d596d0d9d00dcb114e748b
Prasoon Goyal
64d596d0d9d00dcb114e748c
Sattvik Sahai
64d596d0d9d00dcb114e748d
Shaohua Liu
64d596d0d9d00dcb114e748e
Yao Lu
64d596d0d9d00dcb114e748f
Anna Gottardi
64d596d0d9d00dcb114e7490
Shui Hu
64d596d0d9d00dcb114e7491
Yang Liu
64d596d0d9d00dcb114e7492
Dilek Hakkani-Tur
64d596d0d9d00dcb114e7493
Kate Bland
64d596d0d9d00dcb114e7494
Heather Rocker
64d596d0d9d00dcb114e7495
James Jeun
64d596d0d9d00dcb114e7496
Yadunandana Rao
64d596d0d9d00dcb114e7497
Michael Johnston
64d596d0d9d00dcb114e7498
Akshaya Iyengar
64d596d0d9d00dcb114e7499
Arindam Mandal
64d596d0d9d00dcb114e749a
Prem Natarajan
64d596d0d9d00dcb114e749b
Reza Ghanadan
Alexa, play with robot: Introducing the First Alexa Prize SimBot
  Challenge on Embodied AI
The Alexa Prize program has empowered numerous university students to
explore, experiment, and showcase their talents in building conversational
agents through challenges like the SocialBot Grand Challenge and the TaskBot
Challenge. As conversational agents increasingly appear in multimodal and
embodied contexts, it is important to explore the affordances of conversational
interaction augmented with computer vision and physical embodiment. This paper
describes the SimBot Challenge, a new challenge in which university teams
compete to build robot assistants that complete tasks in a simulated physical
environment. This paper provides an overview of the SimBot Challenge, which
included both online and offline challenge phases. We describe the
infrastructure and support provided to the teams including Alexa Arena, the
simulated environment, and the ML toolkit provided to teams to accelerate their
building of vision and language models. We summarize the approaches the
participating teams took to overcome research challenges and extract key
lessons learned. Finally, we provide analysis of the performance of the
competing SimBots during the competition.
University teams competed in the SimBot Challenge to develop robot assistants that completed tasks in a simulated environment, utilizing computer vision and language models.
Alexa Arena simulated environment ML toolkit vision and language models