Workflows Versus Agents in LLM Systems

The term agent has become overloaded. It names everything from a single prompt with a retry loop to a cluster of cooperating processes that negotiate tasks among themselves. The overloading matters because different points on that spectrum carry very different operational costs and very different failure modes. Teams that skip the distinction tend to pay for it in bills, in outages, or in a slow erosion of trust from users who cannot predict how the system will respond to a familiar request.

Anthropic's engineering writeup from late 2024 offers a cleaner vocabulary. It draws a line between workflows, in which large language models and tools are orchestrated through predefined code paths, and agents, in which the model itself dynamically directs its own processes and tool use (Anthropic, 2024). Every design decision in an agentic system eventually traces back to which side of this line the system sits on. The argument of this article is that the default should be a workflow, and that moving to an agent is a conscious step taken against a specific constraint rather than a natural upgrade.

Two shapes, side by side

flowchart LR
    subgraph Workflow
        direction LR
        W_IN([Input]) --> W_S1[LLM step 1]
        W_S1 --> W_S2[LLM step 2]
        W_S2 --> W_OUT([Output])
    end

    subgraph Agent
        direction LR
        A_IN([Input]) --> A_LOOP{LLM decides}
        A_LOOP -->|call tool| A_TOOL[Tool]
        A_TOOL --> A_LOOP
        A_LOOP -->|finish| A_OUT([Output])
    end

In the workflow, control passes from step to step along edges that the author drew. The shape of the computation is visible in the source code. Any reader can read the function top to bottom and know which step runs first, which runs second, and what the handoff between them contains. The model is a component, not a controller.

In the agent, control returns to the model after every observation. The code provides a loop, a set of tools, and a budget, but the sequence of calls is emergent rather than authored. The same input can produce different traces on different runs. The model is the controller.

These are different animals. A workflow is closer to a pipeline in a data system: easier to reason about, easier to test, and easier to cost. An agent is closer to a process that holds a goal and figures out how to pursue it: more flexible, but also more expensive per task and harder to observe.

The workflow in code

A minimal workflow composes model calls as functions. The excerpt below is two steps without any framework. The first call classifies the input. The second call produces a response conditioned on that classification. There is no branching at runtime beyond the author's if statement.

from openai import OpenAI

client = OpenAI()

def workflow(query: str) -> str:
    classification = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Classify as 'billing' or 'technical'. One word."},
            {"role": "user", "content": query},
        ],
    ).choices[0].message.content.strip().lower()

    system = "Answer as a billing specialist." if classification == "billing" \
        else "Answer as a technical support engineer."

    return client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": query}],
    ).choices[0].message.content

The LangChain version makes the sequence explicit and composable, but is identical in intent. The author still draws the edges; the runtime still traverses them as drawn.

from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate

model = init_chat_model("gpt-4o-mini")

classify = ChatPromptTemplate.from_messages([
    ("system", "Classify as 'billing' or 'technical'. One word."),
    ("user", "{query}"),
]) | model

def respond(inputs):
    persona = "billing specialist" if "billing" in inputs["label"].content.lower() \
        else "technical support engineer"
    prompt = ChatPromptTemplate.from_messages([
        ("system", f"Answer as a {persona}."),
        ("user", "{query}"),
    ])
    return (prompt | model).invoke({"query": inputs["query"]})

def workflow(query: str):
    label = classify.invoke({"query": query})
    return respond({"query": query, "label": label}).content

The agent in code

An agent does not spell out the sequence. The model receives a set of tools and a loop budget, and decides at each turn whether to call a tool, respond, or stop. The excerpt below is the raw form. The loop is three lines; the intelligence sits in the model's choices.

def agent(query: str, tools: list, max_steps: int = 10) -> str:
    messages = [{"role": "user", "content": query}]
    for _ in range(max_steps):
        response = client.chat.completions.create(
            model="gpt-4o-mini", messages=messages, tools=tools,
        )
        msg = response.choices[0].message
        if not msg.tool_calls:
            return msg.content
        messages.append(msg)
        for call in msg.tool_calls:
            result = execute_tool(call)
            messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
    return "step budget exhausted"

LangGraph provides a prebuilt ReAct-style agent with the same control flow plus checkpointing, streaming, and interrupt hooks. The reduction in surface code is real; the reduction in operational complexity is not.

from langgraph.prebuilt import create_react_agent
from langchain.chat_models import init_chat_model

agent = create_react_agent(
    model=init_chat_model("gpt-4o-mini"),
    tools=[search_knowledge_base, lookup_customer, escalate_ticket],
)

result = agent.invoke({"messages": [("user", "Why was I charged twice last week?")]})

Runnable versions of both shapes live at github.com/subodhjena/agentic-patterns under examples/01_basic_call.py and the later ReAct examples.

Choosing between them

A short set of conditions covers most real decisions.

Reach for a workflow when the task decomposes into a fixed set of steps that a senior engineer could sketch on a whiteboard in a few minutes. Reach for a workflow when predictability matters more than flexibility: regulated domains, billing, and user-visible flows all benefit from computation whose shape is visible in the source.

Reach for an agent when the input space is too wide to enumerate. Open-ended research, debugging sessions, and problems with unknown tool-call depth are places where the ability to pick the next step at runtime is load-bearing. Reach for an agent when the cost of extra model calls is justified by the value of autonomy. Anthropic notes that agents "trade latency and cost for better performance" and that this trade is worth making only when it is (Anthropic, 2024).

The simplest heuristic available comes from the same source: start with the simplest solution possible, and add complexity only when it improves outcomes. A single well-engineered prompt is a baseline worth measuring against.

The two directions of error

Failure clusters around two mistakes: reaching for autonomy that is not needed, and reaching for structure the problem will not tolerate.

Autonomy without bounds is the first. An agent without a step budget, a cost budget, or a mechanism to halt on repeated failures can loop indefinitely. The observable symptom is a runaway bill or a task that never returns. The fix is a hard ceiling on steps, on tokens, and on tool-call repetition.

Autonomy without evaluation is the second. An agent that cannot be judged against a ground-truth is an agent that cannot be improved. Teams sometimes adopt agentic architectures before they have a means to measure them, and then cannot tell regressions from improvements. Build the evaluation harness before the agent (Anthropic, 2025).

Workflows that model agency are the mirror failure. Teams simulate autonomy with long if chains and nested classifiers. The observable symptom is a function that is easier to rewrite than to read. When the branching is genuinely open-ended, a loop with a tool-calling model is cleaner than a deep decision tree.

Agents that model workflows are the final trap. Full autonomy granted to a problem that in practice only has three outcomes produces high variance on production traces that should have been deterministic. Compressing such problems into a workflow reduces cost and variance at once.

What picking wrong costs

The choice is not abstract. The two shapes pay different bills.

Axis	Workflow	Agent
Cost per task	Predictable, bounded by the number of steps in code	Variable, bounded only by the step budget
Latency	Predictable, often lower	Higher and variable
Reliability	High when the pipeline covers the input space	Depends on model capability and tool design
Observability	Native; each step is a distinct call	Requires tracing to reconstruct the decision path
Flexibility	Limited to the paths the author encoded	Emergent; handles inputs the author did not anticipate
Engineering effort to ship	Higher per path, lower overall	Lower to stand up, higher to operate safely

Most axes favor the structured side. Only flexibility favors the agent, and flexibility is a feature the problem either demands or does not.

Where the distinction echoes through this series

The workflow patterns covered next in this series (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) are all ways of getting more capability out of the workflow side of the line. The agent patterns that follow (ReAct, plan-and-execute, reflexion) are the disciplined ways to live on the agent side. The multi-agent and memory stages add structure to both. Every later article in the series takes a position on the workflow-versus-agent question by implication; it helps to have named the question first.

References

1. Anthropic. Building effective agents. December 2024.
2. Anthropic. A practical guide to building agents. March 2025.
3. OpenAI. Practices for deploying LLM-based agents. 2024.
4. LangChain. LangGraph: building stateful, multi-actor applications with LLMs. 2024.