Designing Production AI Systems with Jev

Jev cannot write the customer response, generate the patch, explain an incident, or produce a SQL query.

It can decide which queue a ticket belongs to, which skill an agent should load, which retrieved passage is relevant, whether an LLM response needs review, or which one of five known actions the runtime should take next.

That makes it quite different from the models we have been building around for the last few years.

TypeSafe released Jev on September 15 as its first System One model. The interface is built around typed questions over some application state. Instead of generating text token by token, Jev returns predefined answers and probabilities that normal software can use directly. TypeSafe currently describes the model as using a new architecture, parallel sampling, and a training method called Reinforcement Learning for Calibrated Decisions. The internal model architecture itself has not been published, so I would not make assumptions about what sits behind the API. (typesafe.ai)

Note on current Jev numbers: Jev is in early access as of September 2026. Pricing, latency, model limits, and benchmark results in this article are based on TypeSafe's current published material unless stated otherwise. TypeSafe has not published Jev's model architecture, parameter count, training data, or enough detail to independently reproduce RLCD. No independent Jev latency or accuracy benchmark had been published as of September 18, 2026.

What matters from the system side is the contract.

You provide state.

You define the possible shape of the answer.

Jev returns a decision distribution.

Code decides what happens after that.

The interface is intentionally narrow

Jev currently exposes three question types.

A Choice asks the model to select from options that are known before the call. Current documentation lists up to 255 choices.

A Score asks for a position on an ordered scale of between 2 and 10 levels.

A Noul asks a yes or no question and returns a probability between 0 and 1.

Several questions can be evaluated against the same state in one request. Current documentation says the questions are evaluated in parallel. The published limits are 64K tokens for the complete request, with 32K available to the state plus the longest individual question. (systemonemodels.org)

Take an incident agent looking at payment failures.

The current state may contain the affected service, recent error samples, deployment information, current incident summary, and the actions available to the agent.

One question could be:

What should the agent do next?

SEARCH_LOGS
SEARCH_CODE
QUERY_INCIDENTS
ASK_HUMAN

Jev might return something conceptually like:

SEARCH_LOGS        0.72
QUERY_INCIDENTS    0.18
SEARCH_CODE        0.08
ASK_HUMAN          0.02

Another question in the same request can ask:

Is there enough evidence to modify production code?

and return:

0.13

Another can score incident severity.

The interesting part for me is not that the result is structured. LLMs can already generate JSON.

The useful difference is that the answer space is part of the model interface. SEARCH_DATABASE_THAT_DOES_NOT_EXIST is not another string the model can decide to generate.

That removes one class of failure.

It does not remove wrong decisions.

If SEARCH_LOGS is the wrong choice, the output is perfectly typed and still wrong. TypeSafe uses language such as zero hallucination in the structural sense that the model cannot generate something outside the declared type. That should not be interpreted as zero semantic error. Their own launch material is clear that uncertainty and probability are still part of the model contract. (typesafe.ai)

The latency is interesting because it changes where a model can sit

TypeSafe currently reports roughly 70 ms to 500 ms end to end latency in its own evaluations. This is a vendor reported range, not a guaranteed production SLO or an independently verified benchmark. TypeSafe also notes that its published runs are generally measured from machines on the US West Coast, where the service is currently based. (typesafe.ai)

I would measure it again from wherever my workload actually runs.

The model latency is only one part of the control loop anyway.

For an agent router, the budget might look roughly like:

state assembly          15 ms
network                  35 ms
Jev                     120 ms
policy evaluation         2 ms
tool dispatch            10 ms

The application sees about 182 ms, not 120 ms.

If state assembly involves three databases and a remote vector search, the 100 ms model no longer gives you a 100 ms decision path.

This matters because Jev is interesting in places where another 3 or 5 second reasoning call changes the shape of the product.

A routing decision between agent steps is one example. Browser control is another. Skill selection, semantic filtering, moderation gates, and real time interface behavior all become more practical when the semantic judgment fits inside hundreds of milliseconds instead of seconds.

There are already early projects using Jev this way. Browser Use has an implementation where Jev selects the browser operation and target element, while a small LLM is called only when text needs to be written. Another project called Foreman puts Jev above slower Codex workers to decide whether requirements were met, testing is sufficient, or human input is needed. (systemonemodels.org)

That hybrid pattern is probably more interesting than Jev replacing an LLM.

The cost becomes meaningful when decisions are everywhere

TypeSafe currently prices Jev at $0.042 per million input tokens, with no metered output token charge. This is current vendor pricing and can change. TypeSafe also acknowledges that it cannot yet prove that the current pricing is not subsidized, so I would not assume the same economics indefinitely when designing a long term capacity model. (typesafe.ai)

For a decision using 1,000 input tokens, the advertised input cost is:

$0.000042

That looks almost irrelevant at one call.

Now imagine 100 million decisions a month with an average state size of 1,000 tokens.

That is 100 billion input tokens.

At the current advertised price:

about $4,200

This is where the model becomes architecturally interesting.

There are workflows where putting a normal generative model into every branch is simply too expensive, especially when the answer being requested is one enum, one score, or one probability.

But there is an easy way to get the economics wrong.

If the flow becomes:

Jev
 ↓
LLM

for every request, then Jev is an additional model call and another network hop.

The cost advantage appears when Jev removes work.

For example, Jev can route 70 percent of requests directly to deterministic code, send 20 percent to a smaller model, and reserve the frontier LLM for the 10 percent that actually needs deeper reasoning.

Or it can filter 40 retrieved passages down to 8 before those passages reach the LLM.

Or it can decide that an agent run is complete without asking another large model to review the whole trace.

That is how I would evaluate the economics.

Not cost per Jev call.

Cost per completed task.

I would put a small decision service around Jev

I would not let every application team scatter TypeSafe API calls and threshold checks throughout the codebase.

Give Jev a small production layer.

The application talks to a decision type, not directly to one model endpoint.

Something like:

DecisionRequest {
    decision_type
    state
    question_set_version
    decision_schema_version
}

The result can carry:

DecisionResult {
    model_version

    answers[]
    probabilities[]

    policy_version
    policy_result
}

This gives one place to handle model version pinning, retries, fallbacks, observability, thresholds, evaluation and calibration.

It also means a decision type can later move from Jev to deterministic code or another model without rewriting every caller.

State construction matters more than it first appears

Jev makes decisions about the state we provide.

I would keep that state much smaller than the total information available to the application.

For the incident agent I might include the current service, recent error patterns, deployment changes, a compact incident summary, the current step, and available actions.

I would not dump the entire conversation, all logs, every previous incident and half the repository into one Jev call because the model has a 64K context window.

Independent documentation tracking Jev's published weaknesses notes that accuracy can degrade as unrelated state grows, along with weaknesses around arithmetic, counting, dates, multiple reasoning hops and prompt injection in the supplied state. (systemonemodels.org)

This is similar to the context layer problem with an LLM.

More available context does not mean all of it belongs in this decision.

For Jev I would probably be even stricter because most decisions should be narrow.

Model routing is one of the cleanest uses

Suppose the product receives a request and has four possible execution paths:

DETERMINISTIC_CODE
SMALL_LLM
FRONTIER_LLM
HUMAN

Jev can answer several questions against the same state.

What is the intent?

How difficult is this?

Does this contain a sensitive operation?

Does this require open ended generation?

Then code maps those probabilities into the execution path.

if sensitive_probability > sensitive_threshold:
    human()

elif complexity == HIGH:
    frontier_llm()

elif intent in deterministic_handlers:
    deterministic_code()

else:
    small_llm()

This is also one of the use cases currently documented around Jev. The decision model identifies intent and difficulty, then normal code chooses a handler. (systemonemodels.org)

The policy belongs in code.

I would not ask Jev:

Which model should I use, GPT, Claude, or code?

and blindly execute the winner.

The model supplies judgments. Software owns policy.

Skill selection is another obvious place

This connects directly to the Agent Skills infrastructure problem.

Assume infrastructure filtering has already reduced 2,000 registered skills to 120 that this agent is actually allowed to use.

Jev can rank those eligible descriptions and separately answer whether any of them really applies.

The boundary matters.

2,000 registered
      |
      v
Infrastructure eligibility
      |
      v
120 allowed
      |
      v
Jev selection
      |
      v
top candidates
      |
      v
Skill loader

Jev should not decide whether a forbidden deployment skill suddenly becomes available to the agent.

It decides among the choices the system has already made eligible.

Agent routing and skill selection are now explicitly listed among the System One use cases. (systemonemodels.org)

RAG filtering is a practical cost use case

A retriever gives the system 40 passages.

Vector similarity tells us that the passages look related to the query. It does not tell us that each passage should enter the final prompt.

Put a judgment stage between retrieval and generation.

Query
  |
Retriever
  |
40 passages
  |
Jev
  |
relevant?
current?
conflicting?
unsafe?
  |
Policy
  |
8 passages
  |
LLM

Now the large model receives less noise and fewer tokens.

System One examples describe both RAG filtering and semantic reranking in this shape. (systemonemodels.org)

There is one thing I would measure carefully.

If Jev sees each passage separately, latency and request count can grow quickly.

If many questions can be evaluated against one shared state, batch where the API shape allows it.

The architecture should save more generation work than it adds in decision work.

Jev around an LLM is probably the pattern I would use most

The two model types do different jobs.

Before the LLM:

Input
  |
Jev
  |
route
risk
intent
context choice
  |
LLM

After the LLM:

LLM Output
    |
    v
   Jev
    |
policy violation?
unsupported claim?
human review?
    |
    v
Policy

Or both:

Input
  |
Jev
  |
LLM
  |
Jev
  |
Action

TypeSafe explicitly positions Jev for verification, guardrails and decisions around LLM workflows, and the independent use case catalog documents the same front and back pattern. (typesafe.ai)

The generative model still investigates the incident, writes the explanation, edits the code and creates the response.

Jev decides things around that work.

That division feels natural to me.

Typed tool dispatch is useful when the arguments are actually bounded

Consider an agent that can choose:

SEARCH_LOGS
SEARCH_CODE
QUERY_INCIDENTS
ASK_HUMAN

That fits a Choice question.

A closed argument such as:

environment =
PRODUCTION
STAGING
DEVELOPMENT

also fits.

A free form SQL query does not.

A patch does not.

A customer email body does not.

The current typed tool dispatch pattern around Jev uses one Choice question for the function and additional typed questions for closed arguments. Code assembles the final tool call. (systemonemodels.org)

This removes failures such as invented enum values or malformed generated JSON for the bounded parts of the call.

It does not magically turn open ended tool arguments into closed set decisions.

The probabilities need policy around them

This is where I would spend more design time than on the API integration.

Suppose Jev returns:

ALLOW_REFUND    0.997
REVIEW          0.002
DENY            0.001

The application still needs to decide what 0.997 means for this action.

A simple policy may use three regions:

>= 0.995
automatic action

0.80 to 0.995
human review

< 0.80
block or gather more evidence

Those numbers are examples only.

A support ticket routing mistake and a $10,000 financial action should not use the same automation threshold.

Thresholds belong in versioned policy.

I would record:

model_version
question_set_version
decision_schema_version
threshold_policy_version

because changing any one of them can change production behavior.

Calibration is what makes probabilities useful

A probability is useful for automation only when the application understands what that probability means on its own traffic.

If Jev says 0.90 on one thousand similar decisions, I want to know how often those decisions were actually correct.

Build calibration curves using production outcomes.

Break them down by decision type, model version, tenant where relevant, input shape and time.

One failure I would watch closely is this:

model confidence = 0.95

historical correctness
around that range = 0.94

then a month later:

model confidence = 0.95

observed correctness = 0.81

Nothing is down.

Latency is normal.

The API returns valid decisions.

Your automation policy has become unsafe.

TypeSafe makes calibration central to Jev's positioning, but I would still validate calibration against my own workload before using the probability as a control boundary. (typesafe.ai)

Measure Jev on your own traffic before choosing automation thresholds. State size, class specific error rates, latency distribution and calibration on your domain matter more than launch averages once Jev controls production behavior.

A few failure modes I would design for immediately

Typed but wrong. The answer is one of the allowed values and still wrong.

High confidence and wrong. This matters most near automatic action thresholds.

Incomplete choice set. If reality requires D and we only give Jev A, B and C, the model cannot create D. Where the domain allows it, include UNKNOWN, NONE, or ESCALATE.

Question drift. Changing "Is this transaction fraudulent?" to "Is there any indication this transaction might be fraudulent?" keeps the same return type and changes the decision boundary. Version questions.

State pollution. Irrelevant or hostile content can change the decision. Keep state task specific and treat user controlled state as untrusted input.

Calibration drift. The probability distribution no longer matches observed outcomes.

Model and policy mismatch. Thresholds were tuned on one model version and the production alias moved to another.

Correlated errors. Five questions answered from the same bad state are not five independent confirmations.

That last one is easy to miss. Parallel answers make the request efficient. They do not make the evidence independent.

I would pin the model version in automated paths

Current public documentation lists jev-1.13.0 as the published model version as of September 2026, with current aliases resolving to it. Aliases can move. (systemonemodels.org)

For an experiment, latest is convenient.

For an automated decision with calibrated thresholds, I would pin the version.

If thresholds were tuned around one model and the alias later points to another model, the HTTP contract can remain identical while the meaning of 0.95 changes.

A model change is a production behavior change even when the response schema does not change.

Roll it like one.

Evaluate it, shadow it, canary it, then move thresholds if the new calibration requires it.

Jev does not belong everywhere

I would not use Jev to write something.

I would not use it for code generation, incident explanation, an open ended plan or a query whose answer space cannot be known before the call.

I would also keep arithmetic, exact counting and date calculations in code. Current published weakness notes for Jev specifically call those areas out, along with multi step reasoning and long irrelevant state. (systemonemodels.org)

If the task is:

Write a migration plan for moving our payment store from Postgres to Spanner.

that is an LLM task.

If the question is:

Which of these five predefined migration stages is this service currently in?

that starts to look like Jev.

That distinction is simple enough to use in architecture reviews.

Observability has to include the decision, policy and outcome

For every meaningful automated Jev decision, I would record:

run_id
step_id

decision_type

model_version
question_set_version
decision_schema_version
policy_version

state_hash

probability_distribution
selected_answer

threshold_used
policy_result

latency
input_tokens
cost

actual_outcome

Then the team can ask useful questions.

How often were decisions in the 0.90 to 0.95 range correct?

Which decision type drifted after the model update?

How many frontier LLM calls did routing avoid?

How often did a human override the Jev recommendation?

What is the cost per successful task with Jev in the loop versus without it?

That is the feedback loop I would care about.

Not whether the API returned HTTP 200.

Current published Jev numbers

These are current TypeSafe or TypeSafe documentation figures, not ArchCrux benchmarks. Verify them against the current API documentation before capacity planning.

TypeSafe has also reported speed advantages as high as 40x to 200x in some of its own evaluations. I would treat those as vendor reported benchmark results rather than general Jev performance characteristics. The detailed task mix, comparison models and production conditions matter too much to use that multiplier as a capacity assumption.

Run your own state sizes.

Measure from your region.

Measure P50, P95 and P99.

Compare task level cost.

Check calibration on your domain.

Then decide which decisions actually belong there.

The invariants I would keep

1. Use Jev when the valid answer space is known before inference.

2. Jev returns judgments. Application code owns control flow and side effects.

3. Typed output removes type and schema failures. It does not remove semantic errors.

4. State is deliberately constructed for the decision instead of filling the context window because capacity exists.

5. Probabilities feed a versioned policy layer rather than directly triggering high consequence actions.

6. Automation thresholds are chosen according to the consequence of being wrong.

7. Calibration is measured against actual outcomes on production like traffic.

8. Model version, question version, decision schema version and threshold policy version are tracked independently.

9. A model upgrade is treated as a behavior change even when the API contract stays the same.

10. Open ended reasoning and generation stay with an LLM or another component designed for them.

11. Deterministic work stays in code.

12. Every automated decision can be traced from the state and Jev probabilities through policy to the actual outcome.