DiffSketchEdit: Text-Guided Vector Sketch Editing

DiffSketchEdit is a powerful tool for editing vector sketches using text instructions. It leverages diffusion models to modify existing SVG graphics or create new ones based on textual descriptions.

Model Description

DiffSketchEdit enables intuitive editing of vector graphics through natural language instructions. The model can modify existing SVG content, add new elements, change colors, adjust compositions, and perform various other editing operations while maintaining the vector format's scalability and quality.

Usage

import requests
import json

# API endpoint
url = "https://api-inference.huggingface.co/models/jree423/diffsketcher_edit"

# Headers
headers = {"Authorization": "Bearer YOUR_HF_TOKEN"}

# Payload for editing existing SVG
payload = {
    "inputs": "add colorful flowers to the scene",
    "parameters": {
        "input_svg": "<svg>...</svg>",  # Optional: existing SVG to edit
        "edit_instruction": "add bright red and yellow flowers",
        "num_paths": 128,
        "num_iter": 300,
        "edit_strength": 0.7,
        "canvas_size": 256
    }
}

# Make request
response = requests.post(url, headers=headers, json=payload)
result = response.json()

# The result contains the edited SVG content
edited_svg = result[0]["svg"]

Parameters

  • input_svg (string, optional): Input SVG content to edit
  • edit_instruction (string): Instruction for how to edit the SVG
  • num_paths (int, default: 128): Number of paths in the edited SVG
  • num_iter (int, default: 300): Number of optimization iterations
  • guidance_scale (float, default: 7.5): Guidance scale for diffusion
  • edit_strength (float, default: 0.7): Strength of the edit (0.0 to 1.0)
  • canvas_size (int, default: 256): Canvas size for SVG generation

Examples

Adding Elements

Input: "add a sun in the sky"
Parameters: {
  "edit_instruction": "add a bright yellow sun in the upper right corner",
  "edit_strength": 0.6
}

Color Changes

Input: "make the flowers red instead of blue"
Parameters: {
  "edit_instruction": "change flower colors from blue to red",
  "edit_strength": 0.8
}

Style Modifications

Input: "make the drawing more abstract"
Parameters: {
  "edit_instruction": "convert to abstract geometric style",
  "edit_strength": 0.9,
  "num_iter": 500
}

Creating New Content

Input: "draw a minimalist landscape"
Parameters: {
  "edit_instruction": "create a simple mountain and tree silhouette",
  "num_paths": 64
}

Features

  • Text-guided editing: Modify SVGs using natural language instructions
  • Flexible editing strength: Control how much the original is changed
  • Preserve vector format: Maintains scalability and editability
  • Creative freedom: Add, remove, or modify any aspect of the design
  • Style transfer: Apply different artistic styles to existing sketches

Use Cases

  • Design iteration: Quickly modify existing vector designs
  • Creative exploration: Experiment with different styles and elements
  • Content adaptation: Adjust graphics for different contexts
  • Collaborative design: Implement feedback through text instructions

Citation

@inproceedings{mohammadrezaei2023diffsketchedit,
  title={DiffSketchEdit: Mask-Free Text-Guided Vector Sketch Editing},
  author={Mohammadrezaei, MohammadHossein and Guo, Hang and Zheng, Yifan and Peng, Xueting and Xu, Humphrey and Shechtman, Eli and Samaras, Dimitris and Xu, Xiaolong},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}

License

This model is released under the MIT License.

Downloads last month
78
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support