DiffSketchEdit: Text-Guided Vector Sketch Editing

DiffSketchEdit is a powerful tool for editing vector sketches using text instructions. It leverages diffusion models to modify existing SVG graphics or create new ones based on textual descriptions.

Model Description

DiffSketchEdit enables intuitive editing of vector graphics through natural language instructions. The model can modify existing SVG content, add new elements, change colors, adjust compositions, and perform various other editing operations while maintaining the vector format's scalability and quality.

Usage

import requests
import json

# API endpoint
url = "https://api-inference.huggingface.co/models/jree423/diffsketcher_edit"

# Headers
headers = {"Authorization": "Bearer YOUR_HF_TOKEN"}

# Payload for editing existing SVG
payload = {
    "inputs": "add colorful flowers to the scene",
    "parameters": {
        "input_svg": "<svg>...</svg>",  # Optional: existing SVG to edit
        "edit_instruction": "add bright red and yellow flowers",
        "num_paths": 128,
        "num_iter": 300,
        "edit_strength": 0.7,
        "canvas_size": 256
    }
}

# Make request
response = requests.post(url, headers=headers, json=payload)
result = response.json()

# The result contains the edited SVG content
edited_svg = result[0]["svg"]

Parameters

input_svg (string, optional): Input SVG content to edit
edit_instruction (string): Instruction for how to edit the SVG
num_paths (int, default: 128): Number of paths in the edited SVG
num_iter (int, default: 300): Number of optimization iterations
guidance_scale (float, default: 7.5): Guidance scale for diffusion
edit_strength (float, default: 0.7): Strength of the edit (0.0 to 1.0)
canvas_size (int, default: 256): Canvas size for SVG generation

Examples

Adding Elements

Input: "add a sun in the sky"
Parameters: {
  "edit_instruction": "add a bright yellow sun in the upper right corner",
  "edit_strength": 0.6
}

Color Changes

Input: "make the flowers red instead of blue"
Parameters: {
  "edit_instruction": "change flower colors from blue to red",
  "edit_strength": 0.8
}

Style Modifications

Input: "make the drawing more abstract"
Parameters: {
  "edit_instruction": "convert to abstract geometric style",
  "edit_strength": 0.9,
  "num_iter": 500
}

Creating New Content

Input: "draw a minimalist landscape"
Parameters: {
  "edit_instruction": "create a simple mountain and tree silhouette",
  "num_paths": 64
}

Features

Text-guided editing: Modify SVGs using natural language instructions
Flexible editing strength: Control how much the original is changed
Preserve vector format: Maintains scalability and editability
Creative freedom: Add, remove, or modify any aspect of the design
Style transfer: Apply different artistic styles to existing sketches

Use Cases

Design iteration: Quickly modify existing vector designs
Creative exploration: Experiment with different styles and elements
Content adaptation: Adjust graphics for different contexts
Collaborative design: Implement feedback through text instructions

Citation

@inproceedings{mohammadrezaei2023diffsketchedit,
  title={DiffSketchEdit: Mask-Free Text-Guided Vector Sketch Editing},
  author={Mohammadrezaei, MohammadHossein and Guo, Hang and Zheng, Yifan and Peng, Xueting and Xu, Humphrey and Shechtman, Eli and Samaras, Dimitris and Xu, Xiaolong},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}

License

This model is released under the MIT License.