The archive
Essays
Deep engineering writing on production AI systems, distributed systems, failure modes, and architecture judgment.
Filtered archive
Essay results
16 essays
MCP vs Agent Skills vs A2A: What Each Does in a Real AI Agent
An AI agent finishes a database-migration review, but the dashboard says the migration ran. Understand how Agent Skills, MCP, A2A, and application state fit together.
Read essayOpenAI Decisions API vs Jev: Designing Systems Around Decisions Instead of Generation
A technical comparison of OpenAI Decisions API and Jev, and how to design a decision layer for routing, tool selection, risk, abstention, calibration and AI agent control.
Read essayWhy Jev Is So Fast: What System One Models Remove from LLM Inference
Why Jev can return decisions in hundreds of milliseconds. A technical look at LLM decode, parallel decisions, latency, throughput, benchmarks, and what TypeSafe has not disclosed.
Read essayDesigning Production AI Systems with Jev
Jev is useful when an AI system needs fast bounded decisions around generative work. The production design still needs state construction, versioned questions, policy, calibration, observability, and clear boundaries around when to act, escalate, or collect more evidence.
Read essayA Skill Is Not a File. It Is a Deploy: Designing Agent Skills Infrastructure
Production Agent Skills are deployments, not files. They need immutable versions, controlled rollout, revocation, trust boundaries, progressive resolution, and a runtime path designed for scale.
Read essayDesigning Identity and Delegation for Production AI Agents
Production AI agents need explicit human identity, managed agent identity, trusted workload identity, narrow delegation, and an evidence chain that survives every authorization hop.
Read essayStateless MCP Didn't Delete Your State: Designing Production MCP Infrastructure
Stateless MCP removes hidden protocol sessions, not application state. Production infrastructure still needs explicit interaction state, durable Tasks, identity, business effect tracking, routing, admission control, and failover.
Read essayDesigning a Production RAG System for 50 Million Documents
A production RAG system at 50 million documents is not only a retrieval problem. The hard parts are ingestion, freshness, permissions, evaluation, observability, cost, and operational control.
Read essayYour LLM Router Can Make Cheap Models Expensive
Why cheap LLM routing can cost more when model switches break cache locality, and how to design a cache aware gateway for agents and chat.
Read essayLLM Inference Is a Scheduler Problem
LLM inference performance is shaped by prefill, decode, KV cache, continuous batching, batching policy, queueing and scheduler tradeoffs, not just model speed.
Read essayYour LLM Rate Limiter Is Counting the Wrong Thing
The system design of LLM Rate Limiter.
Read essayYour LLM Dashboard Is Lying About Load
Request count is a weak signal in LLM systems. Real load comes from tokens, context length, prefill, decode, tools, retries, queues, and tail latency.
Read essayOne Tenant Can Break Your LLM Platform Without Sending Many Requests
The system design of multi-tenant LLM serving.
Read essayYour LLM System Isnt Slow. Your Tail Is Slow.
LLM systems rarely fail because the average request is slow. They fail because the tail gets worse when retrieval, model calls, tools, retries, and streaming delays stack together.
Read essayRetries Make LLM Systems Less Reliable
Retries help normal distributed systems recover from transient failure. In LLM systems, they can also amplify cost, latency, duplicate work, fallback drift and wrong product behavior.
Read essayNewsletter
New essays by email
Email signup is temporarily unavailable. You can still read every essay in the archive.
Explore essays