title: PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with
    LLM
summary: Layout generation is the keystone in achieving automated graphic design,
  requiring arranging the position and size of various multi-modal design
  elements in a visually pleasing and constraint-following manner. Previous
  approaches are either inefficient for large-scale applications or lack
  flexibility for varying design requirements. Our research introduces a unified
  framework for automated graphic layout generation, leveraging the multi-modal
  large language model (MLLM) to accommodate diverse design tasks. In contrast,
  our data-driven method employs structured text (JSON format) and visual
  instruction tuning to generate layouts under specific visual and textual
  constraints, including user-defined natural language specifications. We
  conducted extensive experiments and achieved state-of-the-art (SOTA)
  performance on public multi-modal layout generation benchmarks, demonstrating
  the effectiveness of our method. Moreover, recognizing existing datasets'
  limitations in capturing the complexity of real-world graphic designs, we
  propose two new datasets for much more challenging tasks (user-constrained
  generation and complicated poster), further validating our model's utility in
  real-life settings. Marking by its superior accessibility and adaptability,
  this approach further automates large-scale graphic design tasks. The code and
  datasets will be publicly available on
  https://github.com/posterllava/PosterLLaVA.
paper_authors._id: 666124877259d915ce0fd888
paper_authors.name: Tao Yang
paper_authors._id: 666124877259d915ce0fd889
paper_authors.name: Yingmin Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-06T09:27:36.813Z
paper_authors._id: 65fc18ebd5699b840ddd8fc0
paper_authors.avatarUrl: /avatars/00c3f71b9617c3a8c35212ef90527315.svg
paper_authors.fullname: Yingmin Luo
paper_authors.type: user
paper_authors.user: luoyingmin
paper_authors._id: 666124877259d915ce0fd88a
paper_authors.name: Zhongang Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:40:27.252Z
paper_authors._id: 660cc64f1b14f691990c0ea0
paper_authors.avatarUrl: /avatars/f172d3120b22d745a41cc3f2eb499ce6.svg
paper_authors.fullname: Zhongang Qi
paper_authors.type: user
paper_authors.user: phoenixqza
paper_authors._id: 666124877259d915ce0fd88b
paper_authors.name: Yang Wu
paper_authors._id: 666124877259d915ce0fd88c
paper_authors.name: Ying Shan
paper_authors._id: 666124877259d915ce0fd88d
paper_authors.name: Chang Wen Chen
paper_title: PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with
    LLM
paper_summary: Layout generation is the keystone in achieving automated graphic design,
  requiring arranging the position and size of various multi-modal design
  elements in a visually pleasing and constraint-following manner. Previous
  approaches are either inefficient for large-scale applications or lack
  flexibility for varying design requirements. Our research introduces a unified
  framework for automated graphic layout generation, leveraging the multi-modal
  large language model (MLLM) to accommodate diverse design tasks. In contrast,
  our data-driven method employs structured text (JSON format) and visual
  instruction tuning to generate layouts under specific visual and textual
  constraints, including user-defined natural language specifications. We
  conducted extensive experiments and achieved state-of-the-art (SOTA)
  performance on public multi-modal layout generation benchmarks, demonstrating
  the effectiveness of our method. Moreover, recognizing existing datasets'
  limitations in capturing the complexity of real-world graphic designs, we
  propose two new datasets for much more challenging tasks (user-constrained
  generation and complicated poster), further validating our model's utility in
  real-life settings. Marking by its superior accessibility and adaptability,
  this approach further automates large-scale graphic design tasks. The code and
  datasets will be publicly available on
  https://github.com/posterllava/PosterLLaVA.
paper_ai_keywords: large language model (MLLM) structured text (JSON format) visual instruction tuning state-of-the-art (SOTA) multi-modal layout generation user-defined natural language specifications user-constrained generation complicated poster