First token is one point on a timeline. It does not mean the answer is complete or that the user has learned anything. The scheduler might give each request a quick prefill and first decode, then interleave so many sequences that later tokens arrive slowly. Or the model may get to a tool call faster while the tool queue is now the bottleneck. A research task can also take more model turns because the new route produces poorer plans. I would not infer which from a 35% median alone.

Trace a task from admission to verified completion, with attempt and model-call IDs. For each model turn, separate gateway queue, model admission, prefill, first token, decode gaps, finish reason, and any preemption or KV eviction. Between turns, record tool proposal completion, broker authorization, tool queue and execution, result ingestion, and the next model admission. Include retries, cancellations and human handoffs. vLLM's metrics documentation distinguishes time to first token from inter-token latency and per-request time per output token. Those server metrics are useful, but they do not join three model requests and two tools into one customer task.

Compare old and new on the same assigned task mix, not only completed requests. If the new scheduler makes long decode requests lose service to short ones, p50 first token can improve while p95 completion and abandonment worsen. Plot the distribution by input length, output length, number of turns, tool workflow, tenant and priority. For a single call, approximate completion as wait plus prefill plus first token plus the sum of output gaps, not median first token multiplied by some token count. Correlations matter. The longest calls may be the very ones with the worst decode gaps and most retries.

Now the progress-message twist. If the runtime streams “Searching your documents” at 100 ms before any model output, the product's first-visible-byte metric can improve without a serving change at all. Even if it is a model token, a quick filler phrase is not a meaningful answer. Define first useful, supported response separately from first token. Count time to completed task at an agreed quality threshold, and cost per completed task. An agent that finishes quickly by skipping a required tool check is not a latency win.

Suppose per-token speed is unchanged but tasks still take longer. Look at turn count and tool outcomes. A changed prompt, tool schema, retrieval timeout, or more cautious model may increase calls. If the scheduler change happened at the same time, run a controlled comparison with pinned model, prompt, and tool backend, and a production-shaped mix. A synthetic single-turn load test cannot settle a multi-turn complaint. Conversely, if the same scheduler configuration produces a long decode tail under real concurrency, fix its admission and preemption policy rather than blaming tool latency.

For release I want guardrails on completed useful tasks per unit time, p95 and p99 task completion, abandonment, severe errors and GPU cost, with first token and inter-token metrics as diagnostics. The scheduler can ship to a workload that genuinely benefits while the research-agent route remains pinned, if pools and congestion are isolated enough. Otherwise pause or roll back. The stream starts promptly, then stalls halfway. Where is time going? investigates a stall inside one stream. This question asks why improving a visible early metric can make the whole agent task worse.