title: GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
summary: Despite recent advances in text-to-3D generative methods, there is a notable
  absence of reliable evaluation metrics. Existing metrics usually focus on a
  single criterion each, such as how well the asset aligned with the input text.
  These metrics lack the flexibility to generalize to different evaluation
  criteria and might not align well with human preferences. Conducting user
  preference studies is an alternative that offers both adaptability and
  human-aligned results. User studies, however, can be very expensive to scale.
  This paper presents an automatic, versatile, and human-aligned evaluation
  metric for text-to-3D generative models. To this end, we first develop a prompt
  generator using GPT-4V to generate evaluating prompts, which serve as input to
  compare text-to-3D models. We further design a method instructing GPT-4V to
  compare two 3D assets according to user-defined criteria. Finally, we use these
  pairwise comparison results to assign these models Elo ratings. Experimental
  results suggest our metric strongly align with human preference across
  different evaluation criteria.
paper_authors._id: 659cbaa5e9095ff98daf4b2b
paper_authors.name: Tong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:24:52.104Z
paper_authors._id: 6310929fda0aaad2f70397a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: wutong16
paper_authors._id: 659cbaa5e9095ff98daf4b2c
paper_authors.name: Guandao Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T08:10:36.236Z
paper_authors._id: 64547ca006728ff79a38e7a5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64547ca006728ff79a38e7a5/xfTVQXMSfRv-IbljFpswQ.jpeg
paper_authors.fullname: Guandao Yang
paper_authors.type: user
paper_authors.user: guandao
paper_authors._id: 659cbaa5e9095ff98daf4b2d
paper_authors.name: Zhibing Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T08:10:42.628Z
paper_authors._id: 632a80706813868fa4a649e3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632a80706813868fa4a649e3/MbTsAYGadNwS3-G5vxEnm.jpeg
paper_authors.fullname: Zhibing LI
paper_authors.type: user
paper_authors.user: lizb6626
paper_authors._id: 659cbaa5e9095ff98daf4b2e
paper_authors.name: Kai Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T08:11:28.479Z
paper_authors._id: 63e0a50242591dda0b9dca5c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e0a50242591dda0b9dca5c/c7cBPEBWQDFYimfGnO_SI.png
paper_authors.fullname: Kai Zhang
paper_authors.type: user
paper_authors.user: drogozhang
paper_authors._id: 659cbaa5e9095ff98daf4b2f
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T08:12:17.820Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_authors._id: 659cbaa5e9095ff98daf4b30
paper_authors.name: Leonidas Guibas
paper_authors._id: 659cbaa5e9095ff98daf4b31
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-09T08:12:34.229Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 659cbaa5e9095ff98daf4b32
paper_authors.name: Gordon Wetzstein
paper_title: GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
paper_summary: Despite recent advances in text-to-3D generative methods, there is a notable
  absence of reliable evaluation metrics. Existing metrics usually focus on a
  single criterion each, such as how well the asset aligned with the input text.
  These metrics lack the flexibility to generalize to different evaluation
  criteria and might not align well with human preferences. Conducting user
  preference studies is an alternative that offers both adaptability and
  human-aligned results. User studies, however, can be very expensive to scale.
  This paper presents an automatic, versatile, and human-aligned evaluation
  metric for text-to-3D generative models. To this end, we first develop a prompt
  generator using GPT-4V to generate evaluating prompts, which serve as input to
  compare text-to-3D models. We further design a method instructing GPT-4V to
  compare two 3D assets according to user-defined criteria. Finally, we use these
  pairwise comparison results to assign these models Elo ratings. Experimental
  results suggest our metric strongly align with human preference across
  different evaluation criteria.
paper_ai_summary: An automatic evaluation metric for text-to-3D generative models uses GPT-4V to generate prompts and perform pairwise comparisons for assigning Elo ratings, aligning with human preferences.
paper_ai_keywords: GPT-4V prompt generator pairwise comparison Elo ratings