title: StarVector: Generating Scalable Vector Graphics Code from Images
summary: Scalable Vector Graphics (SVGs) have become integral in modern image
  rendering applications due to their infinite scalability in resolution,
  versatile usability, and editing capabilities. SVGs are particularly popular in
  the fields of web development and graphic design. Existing approaches for SVG
  modeling using deep learning often struggle with generating complex SVGs and
  are restricted to simpler ones that require extensive processing and
  simplification. This paper introduces StarVector, a multimodal SVG generation
  model that effectively integrates Code Generation Large Language Models
  (CodeLLMs) and vision models. Our approach utilizes a CLIP image encoder to
  extract visual representations from pixel-based images, which are then
  transformed into visual tokens via an adapter module. These visual tokens are
  pre-pended to the SVG token embeddings, and the sequence is modeled by the
  StarCoder model using next-token prediction, effectively learning to align the
  visual and code tokens. This enables StarVector to generate unrestricted SVGs
  that accurately represent pixel images. To evaluate StarVector's performance,
  we present SVG-Bench, a comprehensive benchmark for evaluating SVG methods
  across multiple datasets and relevant metrics. Within this benchmark, we
  introduce novel datasets including SVG-Stack, a large-scale dataset of
  real-world SVG examples, and use it to pre-train StarVector as a large
  foundation model for SVGs. Our results demonstrate significant enhancements in
  visual quality and complexity handling over current methods, marking a notable
  advancement in SVG generation technology. Code and models:
  https://github.com/joanrod/star-vector
paper_authors._id: 658256c66259618544fbcd82
paper_authors.name: Juan A. Rodriguez
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-20T08:59:16.000Z
paper_authors._id: 63a614d264f470027818b066
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a614d264f470027818b066/Q5Eih2VqD4NaHB_MkNp32.jpeg
paper_authors.fullname: Juan A. Rodriguez
paper_authors.type: user
paper_authors.user: joanrodai
paper_authors._id: 658256c66259618544fbcd83
paper_authors.name: Shubham Agarwal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:03:30.284Z
paper_authors._id: 617bacfb191221bded6ed2c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652085632597-617bacfb191221bded6ed2c4.jpeg
paper_authors.fullname: Shubham Agarwal
paper_authors.type: user
paper_authors.user: shubhamagarwal92
paper_authors._id: 658256c66259618544fbcd84
paper_authors.name: Issam H. Laradji
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:03:51.069Z
paper_authors._id: 64062855692855e65ae31688
paper_authors.avatarUrl: /avatars/e35d22a037b8b35422d3ee982f133076.svg
paper_authors.fullname: Issam Laradji
paper_authors.type: user
paper_authors.user: issamlaradji
paper_authors._id: 658256c66259618544fbcd85
paper_authors.name: Pau Rodriguez
paper_authors._id: 658256c66259618544fbcd86
paper_authors.name: David Vazquez
paper_authors._id: 658256c66259618544fbcd87
paper_authors.name: Christopher Pal
paper_authors._id: 658256c66259618544fbcd88
paper_authors.name: Marco Pedersoli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:09:49.064Z
paper_authors._id: 64b829042fccad9f5ff20cc7
paper_authors.avatarUrl: /avatars/92cca111ddd0db5d998615c2257a0894.svg
paper_authors.fullname: Marco Pedersoli
paper_authors.type: user
paper_authors.user: Marcopede
paper_title: StarVector: Generating Scalable Vector Graphics Code from Images
paper_summary: Scalable Vector Graphics (SVGs) have become integral in modern image
  rendering applications due to their infinite scalability in resolution,
  versatile usability, and editing capabilities. SVGs are particularly popular in
  the fields of web development and graphic design. Existing approaches for SVG
  modeling using deep learning often struggle with generating complex SVGs and
  are restricted to simpler ones that require extensive processing and
  simplification. This paper introduces StarVector, a multimodal SVG generation
  model that effectively integrates Code Generation Large Language Models
  (CodeLLMs) and vision models. Our approach utilizes a CLIP image encoder to
  extract visual representations from pixel-based images, which are then
  transformed into visual tokens via an adapter module. These visual tokens are
  pre-pended to the SVG token embeddings, and the sequence is modeled by the
  StarCoder model using next-token prediction, effectively learning to align the
  visual and code tokens. This enables StarVector to generate unrestricted SVGs
  that accurately represent pixel images. To evaluate StarVector's performance,
  we present SVG-Bench, a comprehensive benchmark for evaluating SVG methods
  across multiple datasets and relevant metrics. Within this benchmark, we
  introduce novel datasets including SVG-Stack, a large-scale dataset of
  real-world SVG examples, and use it to pre-train StarVector as a large
  foundation model for SVGs. Our results demonstrate significant enhancements in
  visual quality and complexity handling over current methods, marking a notable
  advancement in SVG generation technology. Code and models:
  https://github.com/joanrod/star-vector
paper_ai_keywords: Code Generation Large Language Models (CodeLLMs) CLIP image encoder visual tokens SVG token embeddings StarCoder model next-token prediction SVG-Bench SVG-Stack large foundation model for SVGs