The archive
Essays
Deep engineering writing on production AI systems, distributed systems, failure modes, and architecture judgment.
Filtered archive
Essay results
22 essays
Why Google Built a Computer for Every AI Agent
Google gives AI agents isolated computers without keeping every computer running. A detailed engineering review of microVMs, checkpoint recovery, credentials, memory, and scale.
Read essayMCP vs Agent Skills vs A2A: What Each Does in a Real AI Agent
An AI agent finishes a database-migration review, but the dashboard says the migration ran. Understand how Agent Skills, MCP, A2A, and application state fit together.
Read essayHow Coding Agents Actually Work: Inside the Agent Harness
A technical look inside coding agent harnesses, covering context management, repository search, tools, sandboxes, subagents, durable sessions, recovery, concurrency, completion and evaluation.
Read essayDesigning Containment for Production AI Agents
Production AI agent containment needs more than a sandbox. It needs separate information, execution and effect boundaries, scoped capabilities, egress controls, quarantine, and independent runtime enforcement.
Read essayDesigning Continuous Evaluation for Production AI Agents
Production AI agent evaluation needs more than task success. It needs behavioral checks, trajectory analysis, outcome verification, failure lineage, repeated trials, statistical release gates, and production failures feeding new evaluation evidence.
Read essayA Skill Is Not a File. It Is a Deploy: Designing Agent Skills Infrastructure
Production Agent Skills are deployments, not files. They need immutable versions, controlled rollout, revocation, trust boundaries, progressive resolution, and a runtime path designed for scale.
Read essayAn ALLOW Is Not a Permit: Designing Runtime Authorization for AI Agents
Runtime authorization for AI agents needs more than an ALLOW decision. Production systems need exact action identity, bounded permits, reservations, revocation, serialization, and reconciliation with the resulting business effect.
Read essayStateless MCP Didn't Delete Your State: Designing Production MCP Infrastructure
Stateless MCP removes hidden protocol sessions, not application state. Production infrastructure still needs explicit interaction state, durable Tasks, identity, business effect tracking, routing, admission control, and failover.
Read essayDesigning Observability for Production AI Agents
Production AI agent observability needs more than traces. It needs stable run identity, context manifests, structured telemetry, failure-aware collection, and verified outcomes.
Read essayDesigning the Context Layer for Production AI Agents
Production AI agents need a context layer that assembles evidence, preserves authority and freshness, and separates model reasoning from durable workflow state.
Read essayDesigning Long-Running AI Agents That Survive Failures
Long-running AI agents need durable execution, explicit state, idempotent actions, bounded retries, recovery workflows, and operational control to survive real production failures.
Read essayDesigning a Production RAG System for 50 Million Documents
A production RAG system at 50 million documents is not only a retrieval problem. The hard parts are ingestion, freshness, permissions, evaluation, observability, cost, and operational control.
Read essayYour LLM Router Can Make Cheap Models Expensive
Why cheap LLM routing can cost more when model switches break cache locality, and how to design a cache aware gateway for agents and chat.
Read essayYour LLM Rate Limiter Is Counting the Wrong Thing
The system design of LLM Rate Limiter.
Read essayYour LLM Dashboard Is Lying About Load
Request count is a weak signal in LLM systems. Real load comes from tokens, context length, prefill, decode, tools, retries, queues, and tail latency.
Read essayOne Tenant Can Break Your LLM Platform Without Sending Many Requests
The system design of multi-tenant LLM serving.
Read essayThe Model Is Not the Product
The model is only one unreliable dependency inside a production AI system. The product is the full path around it: retrieval, tools, permissions, latency, evaluation, fallback, and operational control.
Read essayYour LLM System Isnt Slow. Your Tail Is Slow.
LLM systems rarely fail because the average request is slow. They fail because the tail gets worse when retrieval, model calls, tools, retries, and streaming delays stack together.
Read essayRetries Make LLM Systems Less Reliable
Retries help normal distributed systems recover from transient failure. In LLM systems, they can also amplify cost, latency, duplicate work, fallback drift and wrong product behavior.
Read essayYour RAG Evals Are Measuring the Wrong Thing
Most RAG evals measure answer quality at the end. Production failures usually begin much earlier: ingestion, freshness, permissions, retrieval boundaries and generation control.
Read essayRAG in Production Is Mostly Not About Retrieval
Most RAG discussions obsess over retrieval. In production, the harder problems are ingestion, data lifecycle, generation control and evaluation.
Read essayNewsletter
New essays by email
Email signup is temporarily unavailable. You can still read every essay in the archive.
Explore essays