PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes
We introduce the novel task of Language-Guided Object Placement in Real 3D
Scenes. Our model is given a 3D scene's point cloud, a 3D asset, and a textual
prompt broadly describing where the 3D asset should be placed. The task here is
to find a valid placement for the 3D asset that respects the prompt. Compared
with other language-guided localization tasks in 3D scenes such as grounding,
this task has specific challenges: it is ambiguous because it has multiple
valid solutions, and it requires reasoning about 3D geometric relationships and
free space. We inaugurate this task by proposing a new benchmark and evaluation
protocol. We also introduce a new dataset for training 3D LLMs on this task, as
well as the first method to serve as a non-trivial baseline. We believe that
this challenging task and our new benchmark could become part of the suite of
benchmarks used to evaluate and compare generalist 3D LLM models.
681d9dd229119d666079b275
Ahmed Abdelreheem
claimed_verified
2025-05-09T07:21:06.667Z
63a3170f8c0c89dcae316858
https://cdn-avatars.huggingface.co/v1/production/uploads/63a3170f8c0c89dcae316858/aZjVQl1jwnagTDGxalREE.jpeg
Ahmed Abdelreheem
user
Samir55
681d9dd229119d666079b276
Filippo Aleotti
admin_assigned
2025-05-09T09:04:46.129Z
66eae6491cd3794eb4cd1992
/avatars/67e7fa51bfd4331e1b42338af79e6dca.svg
Filippo Aleotti
user
Filippo8
681d9dd229119d666079b277
Jamie Watson
admin_assigned
2025-05-09T09:04:52.606Z
63166685f5e32157c51fe616
/avatars/5b7d8b0e54339d2dc982676af9e4f4fe.svg
Jamie Watson
user
Aileron
681d9dd229119d666079b278
Zawar Qureshi
admin_assigned
2025-05-09T09:04:58.663Z
6703aece547acbd64c531b72
/avatars/8042f99c6eb2c9b61be8c9b950818b2f.svg
Zawar Qureshi
user
zuluquebec
681d9dd229119d666079b279
Abdelrahman Eldesokey
admin_assigned
2025-05-09T09:05:05.828Z
6577f3bacfb2207f11e847bb
/avatars/825998cfebc47d8106f633be5ad10964.svg
Abdelrahman Eldesokey
user
abdo-eldesokey
681d9dd229119d666079b27a
Peter Wonka
681d9dd229119d666079b27b
Gabriel Brostow
681d9dd229119d666079b27c
Sara Vicente
admin_assigned
2025-05-09T09:05:21.990Z
67f517dcbd75b1099bba2857
/avatars/a1a25d7972b1857f8bb49bc9efc02ded.svg
Sara Vicente
user
svicente
681d9dd229119d666079b27d
Guillermo Garcia-Hernando
PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes
We introduce the novel task of Language-Guided Object Placement in Real 3D
Scenes. Our model is given a 3D scene's point cloud, a 3D asset, and a textual
prompt broadly describing where the 3D asset should be placed. The task here is
to find a valid placement for the 3D asset that respects the prompt. Compared
with other language-guided localization tasks in 3D scenes such as grounding,
this task has specific challenges: it is ambiguous because it has multiple
valid solutions, and it requires reasoning about 3D geometric relationships and
free space. We inaugurate this task by proposing a new benchmark and evaluation
protocol. We also introduce a new dataset for training 3D LLMs on this task, as
well as the first method to serve as a non-trivial baseline. We believe that
this challenging task and our new benchmark could become part of the suite of
benchmarks used to evaluate and compare generalist 3D LLM models.
A new task for placing 3D assets in real 3D scenes based on textual prompts is introduced, with a benchmark and dataset for evaluating 3D LLMs.
Language-Guided Object Placement point cloud 3D asset textual prompt valid placement 3D geometric relationships free space benchmark dataset 3D LLMs baseline