StarVector: Generating Scalable Vector Graphics Code from Images
Scalable Vector Graphics (SVGs) have become integral in modern image
rendering applications due to their infinite scalability in resolution,
versatile usability, and editing capabilities. SVGs are particularly popular in
the fields of web development and graphic design. Existing approaches for SVG
modeling using deep learning often struggle with generating complex SVGs and
are restricted to simpler ones that require extensive processing and
simplification. This paper introduces StarVector, a multimodal SVG generation
model that effectively integrates Code Generation Large Language Models
(CodeLLMs) and vision models. Our approach utilizes a CLIP image encoder to
extract visual representations from pixel-based images, which are then
transformed into visual tokens via an adapter module. These visual tokens are
pre-pended to the SVG token embeddings, and the sequence is modeled by the
StarCoder model using next-token prediction, effectively learning to align the
visual and code tokens. This enables StarVector to generate unrestricted SVGs
that accurately represent pixel images. To evaluate StarVector's performance,
we present SVG-Bench, a comprehensive benchmark for evaluating SVG methods
across multiple datasets and relevant metrics. Within this benchmark, we
introduce novel datasets including SVG-Stack, a large-scale dataset of
real-world SVG examples, and use it to pre-train StarVector as a large
foundation model for SVGs. Our results demonstrate significant enhancements in
visual quality and complexity handling over current methods, marking a notable
advancement in SVG generation technology. Code and models:
https://github.com/joanrod/star-vector
658256c66259618544fbcd82
Juan A. Rodriguez
claimed_verified
2023-12-20T08:59:16.000Z
63a614d264f470027818b066
https://cdn-avatars.huggingface.co/v1/production/uploads/63a614d264f470027818b066/Q5Eih2VqD4NaHB_MkNp32.jpeg
Juan A. Rodriguez
user
joanrodai
658256c66259618544fbcd83
Shubham Agarwal
admin_assigned
2023-12-20T09:03:30.284Z
617bacfb191221bded6ed2c4
https://cdn-avatars.huggingface.co/v1/production/uploads/1652085632597-617bacfb191221bded6ed2c4.jpeg
Shubham Agarwal
user
shubhamagarwal92
658256c66259618544fbcd84
Issam H. Laradji
admin_assigned
2023-12-20T09:03:51.069Z
64062855692855e65ae31688
/avatars/e35d22a037b8b35422d3ee982f133076.svg
Issam Laradji
user
issamlaradji
658256c66259618544fbcd85
Pau Rodriguez
658256c66259618544fbcd86
David Vazquez
claimed_verified
2025-05-08T09:00:44.555Z
62f6691f329d4d014d1b4087
https://cdn-avatars.huggingface.co/v1/production/uploads/1660315866736-noauth.jpeg
David Vazquez
user
dvazquez
658256c66259618544fbcd87
Christopher Pal
658256c66259618544fbcd88
Marco Pedersoli
admin_assigned
2023-12-20T09:09:49.064Z
64b829042fccad9f5ff20cc7
/avatars/92cca111ddd0db5d998615c2257a0894.svg
Marco Pedersoli
user
Marcopede
StarVector: Generating Scalable Vector Graphics Code from Images
Scalable Vector Graphics (SVGs) have become integral in modern image
rendering applications due to their infinite scalability in resolution,
versatile usability, and editing capabilities. SVGs are particularly popular in
the fields of web development and graphic design. Existing approaches for SVG
modeling using deep learning often struggle with generating complex SVGs and
are restricted to simpler ones that require extensive processing and
simplification. This paper introduces StarVector, a multimodal SVG generation
model that effectively integrates Code Generation Large Language Models
(CodeLLMs) and vision models. Our approach utilizes a CLIP image encoder to
extract visual representations from pixel-based images, which are then
transformed into visual tokens via an adapter module. These visual tokens are
pre-pended to the SVG token embeddings, and the sequence is modeled by the
StarCoder model using next-token prediction, effectively learning to align the
visual and code tokens. This enables StarVector to generate unrestricted SVGs
that accurately represent pixel images. To evaluate StarVector's performance,
we present SVG-Bench, a comprehensive benchmark for evaluating SVG methods
across multiple datasets and relevant metrics. Within this benchmark, we
introduce novel datasets including SVG-Stack, a large-scale dataset of
real-world SVG examples, and use it to pre-train StarVector as a large
foundation model for SVGs. Our results demonstrate significant enhancements in
visual quality and complexity handling over current methods, marking a notable
advancement in SVG generation technology. Code and models:
https://github.com/joanrod/star-vector
StarVector integrates CodeLLMs and vision models to generate complex SVGs by aligning visual and code tokens, enhancing visual quality and complexity handling.
Code Generation Large Language Models CodeLLMs CLIP image encoder visual tokens SVG token embeddings StarCoder model SVG-Bench SVG-Stack large foundation model SVG generation