PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with
  LLM
Layout generation is the keystone in achieving automated graphic design,
requiring arranging the position and size of various multi-modal design
elements in a visually pleasing and constraint-following manner. Previous
approaches are either inefficient for large-scale applications or lack
flexibility for varying design requirements. Our research introduces a unified
framework for automated graphic layout generation, leveraging the multi-modal
large language model (MLLM) to accommodate diverse design tasks. In contrast,
our data-driven method employs structured text (JSON format) and visual
instruction tuning to generate layouts under specific visual and textual
constraints, including user-defined natural language specifications. We
conducted extensive experiments and achieved state-of-the-art (SOTA)
performance on public multi-modal layout generation benchmarks, demonstrating
the effectiveness of our method. Moreover, recognizing existing datasets'
limitations in capturing the complexity of real-world graphic designs, we
propose two new datasets for much more challenging tasks (user-constrained
generation and complicated poster), further validating our model's utility in
real-life settings. Marking by its superior accessibility and adaptability,
this approach further automates large-scale graphic design tasks. The code and
datasets will be publicly available on
https://github.com/posterllava/PosterLLaVA.
666124877259d915ce0fd888
Tao Yang
666124877259d915ce0fd889
Yingmin Luo
admin_assigned
2024-06-06T09:27:36.813Z
65fc18ebd5699b840ddd8fc0
/avatars/00c3f71b9617c3a8c35212ef90527315.svg
Yingmin Luo
user
luoyingmin
666124877259d915ce0fd88a
Zhongang Qi
claimed_verified
2024-11-19T08:40:27.252Z
660cc64f1b14f691990c0ea0
/avatars/f172d3120b22d745a41cc3f2eb499ce6.svg
Zhongang Qi
user
phoenixqza
666124877259d915ce0fd88b
Yang Wu
666124877259d915ce0fd88c
Ying Shan
666124877259d915ce0fd88d
Chang Wen Chen
PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with
  LLM
Layout generation is the keystone in achieving automated graphic design,
requiring arranging the position and size of various multi-modal design
elements in a visually pleasing and constraint-following manner. Previous
approaches are either inefficient for large-scale applications or lack
flexibility for varying design requirements. Our research introduces a unified
framework for automated graphic layout generation, leveraging the multi-modal
large language model (MLLM) to accommodate diverse design tasks. In contrast,
our data-driven method employs structured text (JSON format) and visual
instruction tuning to generate layouts under specific visual and textual
constraints, including user-defined natural language specifications. We
conducted extensive experiments and achieved state-of-the-art (SOTA)
performance on public multi-modal layout generation benchmarks, demonstrating
the effectiveness of our method. Moreover, recognizing existing datasets'
limitations in capturing the complexity of real-world graphic designs, we
propose two new datasets for much more challenging tasks (user-constrained
generation and complicated poster), further validating our model's utility in
real-life settings. Marking by its superior accessibility and adaptability,
this approach further automates large-scale graphic design tasks. The code and
datasets will be publicly available on
https://github.com/posterllava/PosterLLaVA.
A unified framework using a multi-modal large language model and data-driven methods for layout generation achieves state-of-the-art performance on benchmarks and introduces new datasets for real-world design tasks.
multi-modal large language model data-driven method structured text visual instruction tuning multi-modal layout generation state-of-the-art performance user-defined natural language specifications user-constrained generation complicated poster