title: Theia: Distilling Diverse Vision Foundation Models for Robot Learning
summary: Vision-based robot policy learning, which maps visual inputs to actions,
  necessitates a holistic understanding of diverse visual tasks beyond
  single-task needs like classification or segmentation. Inspired by this, we
  introduce Theia, a vision foundation model for robot learning that distills
  multiple off-the-shelf vision foundation models trained on varied vision tasks.
  Theia's rich visual representations encode diverse visual knowledge, enhancing
  downstream robot learning. Extensive experiments demonstrate that Theia
  outperforms its teacher models and prior robot learning models using less
  training data and smaller model sizes. Additionally, we quantify the quality of
  pre-trained visual representations and hypothesize that higher entropy in
  feature norm distributions leads to improved robot learning performance. Code
  and models are available at https://github.com/bdaiinstitute/theia.
paper_authors._id: 66a8542b26f6962f854fd356
paper_authors.name: Jinghuan Shang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:15:44.374Z
paper_authors._id: 63f686c49cbd6730302cf0f2
paper_authors.avatarUrl: /avatars/b553823be640a07eeb4c7edc2c176d5d.svg
paper_authors.fullname: Jinghuan Shang
paper_authors.type: user
paper_authors.user: Jinghuan
paper_authors._id: 66a8542b26f6962f854fd357
paper_authors.name: Karl Schmeckpeper
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:15:49.919Z
paper_authors._id: 662c012cf5f38dec89bf8ec1
paper_authors.avatarUrl: /avatars/672c96fbf25225be51cd023e8be8e1f9.svg
paper_authors.fullname: Karl Schmeckpeper
paper_authors.type: user
paper_authors.user: kschmeckpeper
paper_authors._id: 66a8542b26f6962f854fd358
paper_authors.name: Brandon B. May
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:16:27.428Z
paper_authors._id: 666c76ad3ae28e1bcfe02c16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/666c76ad3ae28e1bcfe02c16/GG9ImF8QF8BC__7ofbaxO.jpeg
paper_authors.fullname: Brandon B. May
paper_authors.type: user
paper_authors.user: bmay
paper_authors._id: 66a8542b26f6962f854fd359
paper_authors.name: Maria Vittoria Minniti
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:16:33.590Z
paper_authors._id: 665a13ce042dcc43ad3cefec
paper_authors.avatarUrl: /avatars/28a1c0948431eac2e166dcc88468ffd8.svg
paper_authors.fullname: Maria Vittoria Minniti
paper_authors.type: user
paper_authors.user: mminniti-bdai
paper_authors._id: 66a8542b26f6962f854fd35a
paper_authors.name: Tarik Kelestemur
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:16:38.905Z
paper_authors._id: 63bb24c9df5897db7f05047f
paper_authors.avatarUrl: /avatars/80cfd3dc400f9e2dccd275dc54d3e92b.svg
paper_authors.fullname: Tarik Kelestemur
paper_authors.type: user
paper_authors.user: kelestemur
paper_authors._id: 66a8542b26f6962f854fd35b
paper_authors.name: David Watkins
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-30T11:38:47.253Z
paper_authors._id: 66a8cc342c7c3ebdd77652f7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66a8cc342c7c3ebdd77652f7/K9gKXX2ciBcSZOm0IPXiq.jpeg
paper_authors.fullname: David Watkins
paper_authors.type: user
paper_authors.user: davidwatkins
paper_authors._id: 66a8542b26f6962f854fd35c
paper_authors.name: Laura Herlant
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T08:17:08.638Z
paper_authors._id: 66a10bba8645e7fa2629bde6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66a10bba8645e7fa2629bde6/m5ZXO3u1IPABiGCqhABVp.jpeg
paper_authors.fullname: Laura Herlant
paper_authors.type: user
paper_authors.user: lherlant
paper_title: Theia: Distilling Diverse Vision Foundation Models for Robot Learning
paper_summary: Vision-based robot policy learning, which maps visual inputs to actions,
  necessitates a holistic understanding of diverse visual tasks beyond
  single-task needs like classification or segmentation. Inspired by this, we
  introduce Theia, a vision foundation model for robot learning that distills
  multiple off-the-shelf vision foundation models trained on varied vision tasks.
  Theia's rich visual representations encode diverse visual knowledge, enhancing
  downstream robot learning. Extensive experiments demonstrate that Theia
  outperforms its teacher models and prior robot learning models using less
  training data and smaller model sizes. Additionally, we quantify the quality of
  pre-trained visual representations and hypothesize that higher entropy in
  feature norm distributions leads to improved robot learning performance. Code
  and models are available at https://github.com/bdaiinstitute/theia.
paper_ai_keywords: Theia vision foundation model robot learning visual inputs actions off-the-shelf vision foundation models visual representations feature norm distributions entropy