Blog
Writing
Life, tech, and everything in-between.
Harness Design: Planner, Generator, Evaluator for Production LLM Agents
A harness is the set of components around a language model that turn a research prototype into a production system. Anthropic's three-agent architecture (planner, generator, evaluator) mirrors the GAN discriminator-generator dynamic and survives long-running applications.
Agent Evaluation: LLM-as-Judge, Pass-at-K, and Benchmarks
An agent that cannot be measured cannot be improved. Evaluation starts with twenty queries and an LLM-as-judge, scales up through pass-at-k metrics and standard benchmarks, and never trusts any single layer alone.
SubQ and the Indexer Problem
Subquadratic launched yesterday with a 12-million-token model and a $29M seed. The architecture pitch rests on a selector that has to be subquadratic in sequence length. The launch materials describe the property without describing the mechanism.
One Word, Three Jobs: What "Agent" Means at Lunch and in Production
The lunch-table conversation about AI agents and the production-engineering conversation about AI agents are both right. They are talking about three different things and calling all of them "agent."
Guardrails for LLM Agentic Systems: Layered Defense
Safety instructions in a system prompt are not sufficient for production. Benchmark agents still behave unsafely in a significant fraction of cases. The fix is layered defense: prompt-level, tool-level, model-level, and system-level guardrails stacked so any one failure is caught by the next.
Human-in-the-Loop Patterns for LLM Agents
Full autonomy is not the right target for every agent. Put a human in the loop at the points that matter: before dangerous actions, for ambiguous decisions, as an escalation path when the agent is stuck. Checkpointing makes the pauses cheap.
Skills as Contextual Memory: Reusable Procedural Knowledge for LLM Agents
Anthropic's Skills pattern packages reusable procedural knowledge as folders of instructions, scripts, and resources. Progressive disclosure loads only the metadata at startup and brings in the details on demand. Context capacity becomes effectively unbounded.
Persistence and Checkpointing: Time Travel and Recovery for LLM Agents
A long-running agent that loses its state on the next deploy is not a production system. Checkpointing saves agent state after every step, enabling conversational memory, human-in-the-loop pauses, time travel for debugging, and fault-tolerant resumption.
Generative Agents Memory: The Stanford Architecture for Persistent LLM Agents
Stanford's Smallville study ran 25 LLM agents that formed relationships and organized events autonomously. The mechanism that made their behavior coherent over days was a three-part memory system: observation, scored retrieval, and periodic reflection.
Short-Term and Long-Term Memory for LLM Agents
An agent with no memory repeats itself. An agent with bad memory contradicts itself. Getting memory right means splitting it into kinds: short-term for the session, long-term for durable facts, procedural for how-to, episodic for what-happened.
Group Chat Patterns: Round-Robin, Selector, and Swarm for LLM Agents
When several agents share a conversation, the question becomes: who speaks next? Microsoft AutoGen names four ways to decide. Each produces a different coordination style, with different strengths on different workloads.
The Shared Scratchpad: A Common Workspace for Collaborating Agents
When agents need to see what each other are doing, neither a supervisor nor a handoff will do. Give them a shared workspace instead. Every agent reads from and writes to the same state. Context sharing is maximal, but so is the risk of distraction.