BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation
The systematic evaluation and understanding of computer vision models under
varying conditions require large amounts of data with comprehensive and
customized labels, which real-world vision datasets rarely satisfy. While
current synthetic data generators offer a promising alternative, particularly
for embodied AI tasks, they often fall short for computer vision tasks due to
low asset and rendering quality, limited diversity, and unrealistic physical
properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and
assets to generate fully customized synthetic data for systematic evaluation of
computer vision models, based on the newly developed embodied AI benchmark,
BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene
level (e.g., lighting, object placement), the object level (e.g., joint
configuration, attributes such as "filled" and "folded"), and the camera level
(e.g., field of view, focal length). Researchers can arbitrarily vary these
parameters during data generation to perform controlled experiments. We
showcase three example application scenarios: systematically evaluating the
robustness of models across different continuous axes of domain shift,
evaluating scene understanding models on the same set of images, and training
and evaluating simulation-to-real transfer for a novel vision task: unary and
binary state prediction. Project website:
https://behavior-vision-suite.github.io/
66456bea22a71d1eb4ab995f
Yunhao Ge
admin_assigned
2024-05-16T12:53:56.429Z
64370de87cbe1e781fc5c7a2
/avatars/98101739726ed797958e14f3d666933b.svg
Yunhao Ge
user
andyge
66456bea22a71d1eb4ab9960
Yihe Tang
66456bea22a71d1eb4ab9961
Jiashu Xu
admin_assigned
2024-05-16T12:54:31.096Z
62da681ff014388f9086a578
/avatars/a33b0c94c907c8a1179332a4bb1a0165.svg
XUJIASHU
user
JSX
66456bea22a71d1eb4ab9962
Cem Gokmen
admin_assigned
2024-05-16T12:54:37.236Z
6487a3ccaa6b952c784eab16
/avatars/6d543110f169fbc0142b8485e2919f5f.svg
Cem Gokmen
user
cgokmen
66456bea22a71d1eb4ab9963
Chengshu Li
claimed_verified
2024-05-22T07:15:28.719Z
664d3a3079f09472c1193e06
/avatars/3cf4591f36230137bb059e7f2de8bbc3.svg
ChengshuLi
user
chengshu
66456bea22a71d1eb4ab9964
Wensi Ai
66456bea22a71d1eb4ab9965
Benjamin Jose Martinez
admin_assigned
2024-05-16T12:55:01.062Z
63c9d40cbedad7e2bf1a7b79
/avatars/b0a631685cac7aec1252918226e753f4.svg
Benjamin Martinez
user
benjaminmartinez
66456bea22a71d1eb4ab9966
Arman Aydin
admin_assigned
2024-05-16T12:55:14.973Z
62241a3dbe89d35b37211aa0
/avatars/0a77e25cd3f7504c672adf59af61783c.svg
Arman Aydin
user
goldenJester
66456bea22a71d1eb4ab9967
Mona Anvari
admin_assigned
2024-05-16T12:55:21.833Z
63af3f505fe9db73f67b1c4d
/avatars/430ef0638353698b4a08bf6c4aa43f5e.svg
Mona Anvari
user
monaavr
66456bea22a71d1eb4ab9968
Ayush K Chakravarthy
admin_assigned
2024-05-16T12:55:31.341Z
624f9e3d07bd004fb855f5e9
/avatars/86a349cd4053bc0317e27e75a51c69fa.svg
Ayush Chakravarthy
user
ayushchakravarthy
66456bea22a71d1eb4ab9969
Hong-Xing Yu
admin_assigned
2024-05-16T13:24:41.700Z
64ca8dbdbb5d195b99febe28
/avatars/e6eb3ba76a5ced56c8c41506d707d05b.svg
Koven Yu
user
KovenYu
66456bea22a71d1eb4ab996a
Josiah Wong
66456bea22a71d1eb4ab996b
Sanjana Srivastava
66456bea22a71d1eb4ab996c
Sharon Lee
66456bea22a71d1eb4ab996d
Shengxin Zha
66456bea22a71d1eb4ab996e
Laurent Itti
66456bea22a71d1eb4ab996f
Yunzhu Li
admin_assigned
2024-05-16T13:19:13.938Z
64a46562c641afb468a3dc04
/avatars/bc54e5c13f657251b05bf2725086e56d.svg
Yunzhu Li
user
yunzhuli
66456bea22a71d1eb4ab9970
Roberto Martín-Martín
66456bea22a71d1eb4ab9971
Miao Liu
66456bea22a71d1eb4ab9972
Pengchuan Zhang
admin_assigned
2024-05-16T13:22:10.518Z
640056f5261cfa61f39b4c35
/avatars/25aa5dee42caa69327d158a669f6bf3a.svg
Pengchuan Zhang
user
pzzhang
66456bea22a71d1eb4ab9973
Ruohan Zhang
66456bea22a71d1eb4ab9974
Li Fei-Fei
66456bea22a71d1eb4ab9975
Jiajun Wu
admin_assigned
2024-05-16T13:21:34.303Z
647b7b8345616c3de3627863
/avatars/8196364bffed71086ba9a0a4a62027d3.svg
Jiajun Wu
user
jiajunwu
BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation
The systematic evaluation and understanding of computer vision models under
varying conditions require large amounts of data with comprehensive and
customized labels, which real-world vision datasets rarely satisfy. While
current synthetic data generators offer a promising alternative, particularly
for embodied AI tasks, they often fall short for computer vision tasks due to
low asset and rendering quality, limited diversity, and unrealistic physical
properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and
assets to generate fully customized synthetic data for systematic evaluation of
computer vision models, based on the newly developed embodied AI benchmark,
BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene
level (e.g., lighting, object placement), the object level (e.g., joint
configuration, attributes such as "filled" and "folded"), and the camera level
(e.g., field of view, focal length). Researchers can arbitrarily vary these
parameters during data generation to perform controlled experiments. We
showcase three example application scenarios: systematically evaluating the
robustness of models across different continuous axes of domain shift,
evaluating scene understanding models on the same set of images, and training
and evaluating simulation-to-real transfer for a novel vision task: unary and
binary state prediction. Project website:
https://behavior-vision-suite.github.io/
The BEHAVIOR Vision Suite provides tools and assets for generating customized synthetic data to evaluate computer vision models systematically under varying conditions.
computer vision models synthetic data generators embodied AI benchmark BEHAVIOR-1K adjustable parameters scene level object level camera level domain shift scene understanding models simulation-to-real transfer unary state prediction binary state prediction