title: ORES: Open-vocabulary Responsible Visual Synthesis
summary: Avoiding synthesizing specific visual concepts is an essential challenge in
  responsible visual synthesis. However, the visual concept that needs to be
  avoided for responsible visual synthesis tends to be diverse, depending on the
  region, context, and usage scenarios. In this work, we formalize a new task,
  Open-vocabulary Responsible Visual Synthesis (ORES), where the synthesis model
  is able to avoid forbidden visual concepts while allowing users to input any
  desired content. To address this problem, we present a Two-stage Intervention
  (TIN) framework. By introducing 1) rewriting with learnable instruction through
  a large-scale language model (LLM) and 2) synthesizing with prompt intervention
  on a diffusion synthesis model, it can effectively synthesize images avoiding
  any concepts but following the user's query as much as possible. To evaluate on
  ORES, we provide a publicly available dataset, baseline models, and benchmark.
  Experimental results demonstrate the effectiveness of our method in reducing
  risks of image generation. Our work highlights the potential of LLMs in
  responsible visual synthesis. Our code and dataset is public available.
paper_authors._id: 64ede02a74d9b58eabd573d0
paper_authors.name: Minheng Ni
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-08T15:51:52.084Z
paper_authors._id: 6322952915b7beab57c9f439
paper_authors.avatarUrl: /avatars/7584cafb15632d2a05fc9031e3804353.svg
paper_authors.fullname: Minheng Ni
paper_authors.type: user
paper_authors.user: kodenii
paper_authors._id: 64ede02a74d9b58eabd573d1
paper_authors.name: Chenfei Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-29T13:43:37.741Z
paper_authors._id: 63180f0f49af4d408c35f07f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678864054390-63180f0f49af4d408c35f07f.jpeg
paper_authors.fullname: Chenfei Wu
paper_authors.type: user
paper_authors.user: Chenfei
paper_authors._id: 64ede02a74d9b58eabd573d2
paper_authors.name: Xiaodong Wang
paper_authors._id: 64ede02a74d9b58eabd573d3
paper_authors.name: Shengming Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-29T13:45:41.436Z
paper_authors._id: 62430b00b1fc6718f4fa6295
paper_authors.avatarUrl: /avatars/f4554c124ec12ff0457598b5db4b175b.svg
paper_authors.fullname: LanHarmony
paper_authors.type: user
paper_authors.user: LanHarmony
paper_authors._id: 64ede02a74d9b58eabd573d4
paper_authors.name: Lijuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-29T13:46:50.338Z
paper_authors._id: 6413521d4e5305c14f22e110
paper_authors.avatarUrl: /avatars/a6f8d0573e678f79bc3c0b7897b818ce.svg
paper_authors.fullname: Lijuan Wang
paper_authors.type: user
paper_authors.user: Lijuan
paper_authors._id: 64ede02a74d9b58eabd573d5
paper_authors.name: Zicheng Liu
paper_authors._id: 64ede02a74d9b58eabd573d6
paper_authors.name: Nan Duan
paper_title: ORES: Open-vocabulary Responsible Visual Synthesis
paper_summary: Avoiding synthesizing specific visual concepts is an essential challenge in
  responsible visual synthesis. However, the visual concept that needs to be
  avoided for responsible visual synthesis tends to be diverse, depending on the
  region, context, and usage scenarios. In this work, we formalize a new task,
  Open-vocabulary Responsible Visual Synthesis (ORES), where the synthesis model
  is able to avoid forbidden visual concepts while allowing users to input any
  desired content. To address this problem, we present a Two-stage Intervention
  (TIN) framework. By introducing 1) rewriting with learnable instruction through
  a large-scale language model (LLM) and 2) synthesizing with prompt intervention
  on a diffusion synthesis model, it can effectively synthesize images avoiding
  any concepts but following the user's query as much as possible. To evaluate on
  ORES, we provide a publicly available dataset, baseline models, and benchmark.
  Experimental results demonstrate the effectiveness of our method in reducing
  risks of image generation. Our work highlights the potential of LLMs in
  responsible visual synthesis. Our code and dataset is public available.
paper_ai_keywords: Open-vocabulary Responsible Visual Synthesis (ORES) Two-stage Intervention (TIN) framework rewriting with learnable instruction large-scale language model (LLM) synthesizing with prompt intervention diffusion synthesis model dataset baseline models benchmark image generation