title: LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language
    Models
summary: Graphic layout generation, a growing research field, plays a significant role
  in user engagement and information perception. Existing methods primarily treat
  layout generation as a numerical optimization task, focusing on quantitative
  aspects while overlooking the semantic information of layout, such as the
  relationship between each layout element. In this paper, we propose LayoutNUWA,
  the first model that treats layout generation as a code generation task to
  enhance semantic information and harness the hidden layout expertise of large
  language models~(LLMs). More concretely, we develop a Code Instruct Tuning
  (CIT) approach comprising three interconnected modules: 1) the Code
  Initialization (CI) module quantifies the numerical conditions and initializes
  them as HTML code with strategically placed masks; 2) the Code Completion (CC)
  module employs the formatting knowledge of LLMs to fill in the masked portions
  within the HTML code; 3) the Code Rendering (CR) module transforms the
  completed code into the final layout output, ensuring a highly interpretable
  and transparent layout generation procedure that directly maps code to a
  visualized layout. We attain significant state-of-the-art performance (even
  over 50\% improvements) on multiple datasets, showcasing the strong
  capabilities of LayoutNUWA. Our code is available at
  https://github.com/ProjectNUWA/LayoutNUWA.
paper_authors._id: 6509112b623330a3a5104b81
paper_authors.name: Zecheng Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-20T13:16:43.387Z
paper_authors._id: 64096ef79e9f790c905b846d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64096ef79e9f790c905b846d/hVzw656lXdzbCxTtnheud.jpeg
paper_authors.fullname: Zecheng Tang
paper_authors.type: user
paper_authors.user: ZetangForward
paper_authors._id: 6509112b623330a3a5104b82
paper_authors.name: Chenfei Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-20T14:23:07.118Z
paper_authors._id: 63180f0f49af4d408c35f07f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678864054390-63180f0f49af4d408c35f07f.jpeg
paper_authors.fullname: Chenfei Wu
paper_authors.type: user
paper_authors.user: Chenfei
paper_authors._id: 6509112b623330a3a5104b83
paper_authors.name: Juntao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-20T14:23:22.259Z
paper_authors._id: 60d1d7f94672e8c13e7c2bce
paper_authors.avatarUrl: /avatars/be3a3990492ef1c5f3ff33cc043ae166.svg
paper_authors.fullname: Juntao Li
paper_authors.type: user
paper_authors.user: ljtsuda
paper_authors._id: 6509112b623330a3a5104b84
paper_authors.name: Nan Duan
paper_title: LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language
    Models
paper_summary: Graphic layout generation, a growing research field, plays a significant role
  in user engagement and information perception. Existing methods primarily treat
  layout generation as a numerical optimization task, focusing on quantitative
  aspects while overlooking the semantic information of layout, such as the
  relationship between each layout element. In this paper, we propose LayoutNUWA,
  the first model that treats layout generation as a code generation task to
  enhance semantic information and harness the hidden layout expertise of large
  language models~(LLMs). More concretely, we develop a Code Instruct Tuning
  (CIT) approach comprising three interconnected modules: 1) the Code
  Initialization (CI) module quantifies the numerical conditions and initializes
  them as HTML code with strategically placed masks; 2) the Code Completion (CC)
  module employs the formatting knowledge of LLMs to fill in the masked portions
  within the HTML code; 3) the Code Rendering (CR) module transforms the
  completed code into the final layout output, ensuring a highly interpretable
  and transparent layout generation procedure that directly maps code to a
  visualized layout. We attain significant state-of-the-art performance (even
  over 50\% improvements) on multiple datasets, showcasing the strong
  capabilities of LayoutNUWA. Our code is available at
  https://github.com/ProjectNUWA/LayoutNUWA.