title: GlyphControl: Glyph Conditional Control for Visual Text Generation
summary: Recently, there has been a growing interest in developing diffusion-based
  text-to-image generative models capable of generating coherent and well-formed
  visual text. In this paper, we propose a novel and efficient approach called
  GlyphControl to address this task. Unlike existing methods that rely on
  character-aware text encoders like ByT5 and require retraining of text-to-image
  models, our approach leverages additional glyph conditional information to
  enhance the performance of the off-the-shelf Stable-Diffusion model in
  generating accurate visual text. By incorporating glyph instructions, users can
  customize the content, location, and size of the generated text according to
  their specific requirements. To facilitate further research in visual text
  generation, we construct a training benchmark dataset called LAION-Glyph. We
  evaluate the effectiveness of our approach by measuring OCR-based metrics and
  CLIP scores of the generated visual text. Our empirical evaluations demonstrate
  that GlyphControl outperforms the recent DeepFloyd IF approach in terms of OCR
  accuracy and CLIP scores, highlighting the efficacy of our method.
paper_authors._id: 647568aca855203d8fef2728
paper_authors.name: Yukang Yang
paper_authors._id: 647568aca855203d8fef2729
paper_authors.name: Dongnan Gui
paper_authors._id: 647568aca855203d8fef272a
paper_authors.name: Yuhui Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T10:08:04.279Z
paper_authors._id: 646f69a6041e48e1c4728de3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646f69a6041e48e1c4728de3/U5OaW6PgsXTXnfG03xs9Q.png
paper_authors.fullname: GlyphByT5
paper_authors.type: user
paper_authors.user: GlyphByT5
paper_authors._id: 647568aca855203d8fef272b
paper_authors.name: Haisong Ding
paper_authors._id: 647568aca855203d8fef272c
paper_authors.name: Han Hu
paper_authors._id: 647568aca855203d8fef272d
paper_authors.name: Kai Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-05-21T09:22:22.961Z
paper_authors._id: 664c6083f48f9e269c593105
paper_authors.avatarUrl: /avatars/8cd69f3811202b6e2a1cb13ee53a8148.svg
paper_authors.fullname: Kai Chen
paper_authors.type: user
paper_authors.user: chenkagi
paper_title: GlyphControl: Glyph Conditional Control for Visual Text Generation
paper_summary: Recently, there has been a growing interest in developing diffusion-based
  text-to-image generative models capable of generating coherent and well-formed
  visual text. In this paper, we propose a novel and efficient approach called
  GlyphControl to address this task. Unlike existing methods that rely on
  character-aware text encoders like ByT5 and require retraining of text-to-image
  models, our approach leverages additional glyph conditional information to
  enhance the performance of the off-the-shelf Stable-Diffusion model in
  generating accurate visual text. By incorporating glyph instructions, users can
  customize the content, location, and size of the generated text according to
  their specific requirements. To facilitate further research in visual text
  generation, we construct a training benchmark dataset called LAION-Glyph. We
  evaluate the effectiveness of our approach by measuring OCR-based metrics and
  CLIP scores of the generated visual text. Our empirical evaluations demonstrate
  that GlyphControl outperforms the recent DeepFloyd IF approach in terms of OCR
  accuracy and CLIP scores, highlighting the efficacy of our method.
paper_ai_keywords: diffusion-based text-to-image generative models GlyphControl character-aware text encoders ByT5 Stable-Diffusion model glyph conditional information LAION-Glyph OCR-based metrics CLIP scores visual text generation