Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code
  Workflows
Large Language Models (LLMs) such as GPT-4o can handle a wide range of
complex tasks with the right prompt. As per token costs are reduced, the
advantages of fine-tuning Small Language Models (SLMs) for real-world
applications -- faster inference, lower costs -- may no longer be clear. In
this work, we present evidence that, for domain-specific tasks that require
structured outputs, SLMs still have a quality advantage. We compare fine-tuning
an SLM against prompting LLMs on the task of generating low-code workflows in
JSON form. We observe that while a good prompt can yield reasonable results,
fine-tuning improves quality by 10% on average. We also perform systematic
error analysis to reveal model limitations.
683da3efae07a09d565d638e
Orlando Marquez Ayala
683da3efae07a09d565d638f
Patrice Bechard
claimed_verified
2025-06-03T08:49:25.807Z
607f060442beb4da0f990182
https://cdn-avatars.huggingface.co/v1/production/uploads/607f060442beb4da0f990182/j5W2tLyU6JqkaTf3kv66s.jpeg
Patrice Bechard
user
patricebechard
683da3efae07a09d565d6390
Emily Chen
683da3efae07a09d565d6391
Maggie Baird
683da3efae07a09d565d6392
Jingfei Chen
Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code
  Workflows
Large Language Models (LLMs) such as GPT-4o can handle a wide range of
complex tasks with the right prompt. As per token costs are reduced, the
advantages of fine-tuning Small Language Models (SLMs) for real-world
applications -- faster inference, lower costs -- may no longer be clear. In
this work, we present evidence that, for domain-specific tasks that require
structured outputs, SLMs still have a quality advantage. We compare fine-tuning
an SLM against prompting LLMs on the task of generating low-code workflows in
JSON form. We observe that while a good prompt can yield reasonable results,
fine-tuning improves quality by 10% on average. We also perform systematic
error analysis to reveal model limitations.
Fine-tuning Small Language Models yields higher quality for structured output tasks compared to prompting Large Language Models, despite reduced token costs.
Large Language Models GPT-4o Small Language Models fine-tuning low-code workflows JSON error analysis