title: CoSTAast: Cost-Sensitive Toolpath Agent for Multi-turn Image Editing
summary: Text-to-image models like stable diffusion and DALLE-3 still struggle with
  multi-turn image editing. We decompose such a task as an agentic workflow
  (path) of tool use that addresses a sequence of subtasks by AI tools of varying
  costs. Conventional search algorithms require expensive exploration to find
  tool paths. While large language models (LLMs) possess prior knowledge of
  subtask planning, they may lack accurate estimations of capabilities and costs
  of tools to determine which to apply in each subtask. Can we combine the
  strengths of both LLMs and graph search to find cost-efficient tool paths? We
  propose a three-stage approach "CoSTA*" that leverages LLMs to create a subtask
  tree, which helps prune a graph of AI tools for the given task, and then
  conducts A* search on the small subgraph to find a tool path. To better balance
  the total cost and quality, CoSTA* combines both metrics of each tool on every
  subtask to guide the A* search. Each subtask's output is then evaluated by a
  vision-language model (VLM), where a failure will trigger an update of the
  tool's cost and quality on the subtask. Hence, the A* search can recover from
  failures quickly to explore other paths. Moreover, CoSTA* can automatically
  switch between modalities across subtasks for a better cost-quality trade-off.
  We build a novel benchmark of challenging multi-turn image editing, on which
  CoSTA* outperforms state-of-the-art image-editing models or agents in terms of
  both cost and quality, and performs versatile trade-offs upon user preference.
paper_authors._id: 67d393ca336d57afb21bbf63
paper_authors.name: Advait Gupta
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:36.855Z
paper_authors._id: 67a99ec47b754f038d110926
paper_authors.avatarUrl: /avatars/e1ff318a42ccb75b094bbe7dae0cabec.svg
paper_authors.fullname: Advait Gupta
paper_authors.type: user
paper_authors.user: advaitgupta
paper_authors._id: 67d393ca336d57afb21bbf64
paper_authors.name: NandaKiran Velaga
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:34.327Z
paper_authors._id: 672f89e6d7f4171f374dacea
paper_authors.avatarUrl: /avatars/4a8b378e13e862586bb428fdf000b3cc.svg
paper_authors.fullname: NandaKiran Velaga
paper_authors.type: user
paper_authors.user: nandakiran09
paper_authors._id: 67d393ca336d57afb21bbf65
paper_authors.name: Dang Nguyen
paper_authors._id: 67d393ca336d57afb21bbf66
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:39.157Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_title: CoSTAast: Cost-Sensitive Toolpath Agent for Multi-turn Image Editing
paper_summary: Text-to-image models like stable diffusion and DALLE-3 still struggle with
  multi-turn image editing. We decompose such a task as an agentic workflow
  (path) of tool use that addresses a sequence of subtasks by AI tools of varying
  costs. Conventional search algorithms require expensive exploration to find
  tool paths. While large language models (LLMs) possess prior knowledge of
  subtask planning, they may lack accurate estimations of capabilities and costs
  of tools to determine which to apply in each subtask. Can we combine the
  strengths of both LLMs and graph search to find cost-efficient tool paths? We
  propose a three-stage approach "CoSTA*" that leverages LLMs to create a subtask
  tree, which helps prune a graph of AI tools for the given task, and then
  conducts A* search on the small subgraph to find a tool path. To better balance
  the total cost and quality, CoSTA* combines both metrics of each tool on every
  subtask to guide the A* search. Each subtask's output is then evaluated by a
  vision-language model (VLM), where a failure will trigger an update of the
  tool's cost and quality on the subtask. Hence, the A* search can recover from
  failures quickly to explore other paths. Moreover, CoSTA* can automatically
  switch between modalities across subtasks for a better cost-quality trade-off.
  We build a novel benchmark of challenging multi-turn image editing, on which
  CoSTA* outperforms state-of-the-art image-editing models or agents in terms of
  both cost and quality, and performs versatile trade-offs upon user preference.
paper_ai_summary: A three-stage approach, CoSTA*, combines LLMs and A* search to efficiently find cost-effective tool paths for multi-turn text-to-image editing, outperforming state-of-the-art models in cost and quality.
paper_ai_keywords: stable diffusion DALLE-3 multi-turn image editing agentic workflow tool use subtasks large language models (LLMs) graph search A* search subtask tree subgraph vision-language model (VLM) cost-efficient tool paths cost-quality trade-off multi-turn image editing benchmark