Prompt Chaining: Sequential LLM Pipelines with Validation Gates

A single well-scoped LLM call is a reliable unit of work. A single unscoped call asked to plan, draft, validate, and polish in one pass is a coin toss. The latter produces output that is usually plausible and occasionally incoherent, with no obvious point at which the harness can intervene. Prompt chaining is the simplest workflow pattern that fixes this. Decompose the task into a fixed sequence of LLM calls; optionally insert programmatic validation gates between them.

Anthropic places prompt chaining at the base of the workflow hierarchy and recommends it whenever the task decomposes cleanly into steps that are each simpler than the whole (Anthropic, 2024). This is the first pattern to reach for when a one-shot prompt becomes unreliable and the task has clear structure. The feature that distinguishes a chain from a plain sequence of calls is the gate; the rest of this article turns on it.

Anatomy of a chain

flowchart LR
    IN([Input]) --> S1[Step 1: LLM call]
    S1 --> G1{Gate 1}
    G1 -->|valid| S2[Step 2: LLM call]
    G1 -->|invalid| R1[Retry or fail]
    S2 --> G2{Gate 2}
    G2 -->|valid| S3[Step 3: LLM call]
    G2 -->|invalid| R2[Retry or fail]
    S3 --> OUT([Output])

Each step handles one subtask. Each step receives a focused prompt, a narrow context, and produces a known shape of output. The next step consumes that output, applies its own focused prompt, and produces the next shape. The sequence is declared by the author; nothing in the chain decides at runtime to skip, reorder, or invoke a step that does not appear in the code. That predictability is the reason to use a chain rather than an agent.

The gate is the feature

A gate is a programmatic check that inspects the output of one step before allowing the next to run. A gate can verify schema, enforce length, check policy, or call a smaller model to judge quality. If the gate fails, the chain either retries the previous step, falls back to a safe default, or surfaces an error. The gate keeps a bad intermediate result from propagating through the rest of the pipeline.

A chain without gates is a function composition. It works, but it does not earn the pattern's name. The cases where chaining adds value over a one-shot prompt are the cases where a bad intermediate output is worth catching before it contaminates a later step. That is almost always.

Three steps in code

A chain is usually three to five steps. Longer chains are possible; every additional step adds a latency budget, a point of failure, and a place for context to drift. The design principle is to stop at the shortest chain that produces reliable output.

The excerpt below is three steps without a framework. The task is to write a short marketing post about a new feature. Step one generates an outline. A gate validates that the outline has the expected number of sections. Step two drafts the post from the outline. Step three polishes the draft.

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class Outline(BaseModel):
    title: str
    sections: list[str]

def generate_outline(feature: str) -> Outline:
    return client.responses.parse(
        model="gpt-4o-mini",
        instructions="Produce an outline with a title and 3 section headings.",
        input=feature, text_format=Outline,
    ).output[0].content[0].parsed

def outline_is_valid(outline: Outline) -> bool:
    return 3 <= len(outline.sections) <= 5 and bool(outline.title.strip())

def draft(outline: Outline) -> str:
    return client.responses.create(
        model="gpt-4o-mini",
        instructions="Write a 300-word post from this outline.",
        input=outline.model_dump_json(),
    ).output_text

def polish(draft_text: str) -> str:
    return client.responses.create(
        model="gpt-4o-mini",
        instructions="Polish the post. Tighten prose. Keep under 300 words.",
        input=draft_text,
    ).output_text

def write_post(feature: str) -> str:
    outline = generate_outline(feature)
    if not outline_is_valid(outline):
        outline = generate_outline(feature)
    return polish(draft(outline))

The LangGraph version declares the chain as a state graph. The state carries intermediate values between nodes. Conditional edges express gates. The runtime executes the graph in topological order.

from typing import TypedDict, Literal
from langgraph.graph import StateGraph, START, END
from langchain.chat_models import init_chat_model

model = init_chat_model("gpt-4o-mini")

class State(TypedDict):
    feature: str
    outline: Outline
    draft: str
    final: str

def outline_node(s: State) -> State:
    return {**s, "outline": model.with_structured_output(Outline)
            .invoke(f"Outline a post about: {s['feature']}. 3 sections.")}

def gate(s: State) -> Literal["draft", "outline"]:
    return "draft" if outline_is_valid(s["outline"]) else "outline"

def draft_node(s: State) -> State:
    return {**s, "draft": model.invoke(f"Write 300 words from: {s['outline']}").content}

def polish_node(s: State) -> State:
    return {**s, "final": model.invoke(f"Polish, keep under 300 words: {s['draft']}").content}

graph = (StateGraph(State)
         .add_node("outline", outline_node).add_node("draft", draft_node)
         .add_node("polish", polish_node)
         .add_edge(START, "outline")
         .add_conditional_edges("outline", gate, {"draft": "draft", "outline": "outline"})
         .add_edge("draft", "polish").add_edge("polish", END)
         .compile())

Both versions make the shape of the computation visible in the source. A reader can trace the sequence top to bottom without running the code. Runnable forms live at github.com/subodhjena/agentic-patterns under examples/04_prompt_chaining.py.

When a chain becomes something else

Chaining is not a universal solution. The failure modes below are signals that a different pattern is the right response.

The task does not decompose cleanly. If every step depends on information that only emerges later, the chain is fighting the shape of the problem. Reach for an agent that can revisit earlier decisions.

The chain becomes a tree in disguise. If gates start branching into many recovery paths, the pattern is no longer chaining; it is routing. Switch to an explicit routing pattern, covered in the next article.

Cumulative drift. Each step introduces small errors; a long chain amplifies them. The fix is shorter chains or better gates. A five-step chain with no validation almost always underperforms a three-step chain with a gate.

Latency stacks. N steps cost N round-trips plus N decoding times. When latency matters, consolidate adjacent steps that a single strong model can handle at once.

Context bloat. Each step may retain prior outputs, and the final step can end up receiving more context than it needs. Prune the state between steps. The chain is not obligated to pass everything forward.

Gates that never fire. A validation gate that passes every time is decoration. Either remove it or tighten its conditions until it catches real failures.

What the trade actually buys

The choice between a chain and a one-shot prompt trades latency and cost for reliability and observability.

Axis	One-shot prompt	Prompt chain
Latency	Low, one round-trip	Higher, N round-trips
Cost	Low	Higher, N calls
Accuracy on structured tasks	Variable, often lower	Higher, each step is simpler
Observability	Single opaque response	Distinct intermediate values
Failure isolation	Hard, failures are opaque	Easy, failure attributable to a step
Flexibility at runtime	None beyond the prompt	None beyond the chain

A chain gives up runtime flexibility. If the problem genuinely needs to adapt its sequence to the input, an agent is the right answer. When the sequence is stable, a chain is almost always the better choice.

Neighbors in the series

Routing, the next article, extends chaining by dispatching to one of several sub-chains based on the input. Evaluator-optimizer resembles a chain with a loop-back gate that iterates on the output until an evaluator is satisfied. Plan-and-execute, covered in the Agents stage, is a chain whose steps are decided at runtime rather than in source. Context engineering, the previous article, applies to every step of the chain: each step is a separate window with a narrower context than a one-shot call would use.

References

1. Anthropic. Building effective agents. December 2024.
2. OpenAI. Agents guide: composing workflows. 2024.
3. LangChain. LangGraph: StateGraph and edges. 2024.
4. Schick, Timo, et al. Toolformer: Language Models Can Teach Themselves to Use Tools. 2023.
5. Wu, Qingyun, et al. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. 2023.