title: MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills
summary: Retouching is an essential task in post-manipulation of raw photographs.
  Generative editing, guided by text or strokes, provides a new tool accessible
  to users but can easily change the identity of the original objects in
  unacceptable and unpredictable ways. In contrast, although traditional
  procedural edits, as commonly supported by photoediting tools (e.g., Gimp,
  Lightroom), are conservative, they are still preferred by professionals.
  Unfortunately, professional quality retouching involves many individual
  procedural editing operations that is challenging to plan for most novices. In
  this paper, we ask if a multimodal large language model (MLLM) can be taught to
  critique raw photographs, suggest suitable remedies, and finally realize them
  with a given set of pre-authored procedural image operations. We demonstrate
  that MLLMs can be first made aware of the underlying image processing
  operations, by training them to solve specially designed visual puzzles.
  Subsequently, such an operation-aware MLLM can both plan and propose edit
  sequences. To facilitate training, given a set of expert-edited photos, we
  synthesize a reasoning dataset by procedurally manipulating the expert edits
  and then grounding a pretrained LLM on the visual adjustments, to synthesize
  reasoning for finetuning. The proposed retouching operations are, by
  construction, understandable by the users, preserve object details and
  resolution, and can be optionally overridden. We evaluate our setup on a
  variety of test examples and show advantages, in terms of explainability and
  identity preservation, over existing generative and other procedural
  alternatives. Code, data, models, and supplementary results can be found via
  our project website at https://monetgpt.github.io.
paper_authors._id: 68233e2a952abfa2afb65c6d
paper_authors.name: Niladri Shekhar Dutt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-14T07:35:51.761Z
paper_authors._id: 65b3e258d5bf0d622a89701b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b3e258d5bf0d622a89701b/Bv2Fv8Hyc6w5tvNCPTg2_.jpeg
paper_authors.fullname: Niladri Shekhar Dutt
paper_authors.type: user
paper_authors.user: niladridutt
paper_authors._id: 68233e2a952abfa2afb65c6e
paper_authors.name: Duygu Ceylan
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-13T12:42:24.105Z
paper_authors._id: 632322ca2c9469d56c27e1a3
paper_authors.avatarUrl: /avatars/55b12a72bf67a401bcc4fd4b4fafef80.svg
paper_authors.fullname: Duygu Ceylan
paper_authors.type: user
paper_authors.user: duyguceylan
paper_authors._id: 68233e2a952abfa2afb65c6f
paper_authors.name: Niloy J. Mitra
paper_title: MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills
paper_summary: Retouching is an essential task in post-manipulation of raw photographs.
  Generative editing, guided by text or strokes, provides a new tool accessible
  to users but can easily change the identity of the original objects in
  unacceptable and unpredictable ways. In contrast, although traditional
  procedural edits, as commonly supported by photoediting tools (e.g., Gimp,
  Lightroom), are conservative, they are still preferred by professionals.
  Unfortunately, professional quality retouching involves many individual
  procedural editing operations that is challenging to plan for most novices. In
  this paper, we ask if a multimodal large language model (MLLM) can be taught to
  critique raw photographs, suggest suitable remedies, and finally realize them
  with a given set of pre-authored procedural image operations. We demonstrate
  that MLLMs can be first made aware of the underlying image processing
  operations, by training them to solve specially designed visual puzzles.
  Subsequently, such an operation-aware MLLM can both plan and propose edit
  sequences. To facilitate training, given a set of expert-edited photos, we
  synthesize a reasoning dataset by procedurally manipulating the expert edits
  and then grounding a pretrained LLM on the visual adjustments, to synthesize
  reasoning for finetuning. The proposed retouching operations are, by
  construction, understandable by the users, preserve object details and
  resolution, and can be optionally overridden. We evaluate our setup on a
  variety of test examples and show advantages, in terms of explainability and
  identity preservation, over existing generative and other procedural
  alternatives. Code, data, models, and supplementary results can be found via
  our project website at https://monetgpt.github.io.
paper_ai_summary: A multimodal large language model is trained to suggest and apply procedural edits to photographs, preserving object identity and providing explainable results.
paper_ai_keywords: multimodal large language model MLLM procedural editing operation-aware visual puzzles reasoning dataset pretrained LLM image processing operations explainability identity preservation