An engineering agent gets a request. Payment failures are up. Investigate and prepare a remediation pull request.

The organization has a skill for this. payment_incident_investigation, version 17. It holds the procedure, two query scripts, a reference to the payment runbook, and a dependency on the production observability MCP server.

Version 18 is rolling out. It changes the incident query, needs a newer observability schema, and adds a script. It has passed tests. It has not seen real traffic.

Some agents are mid run on 17. New runs are starting to get 18. And at some point someone is going to find a problem in one of them.

Most teams treat a skill as a folder with a SKILL.md in it. That is fine for a laptop. It is not fine for ten thousand agents.

A skill is a deploy. The moment you accept that, you inherit every problem a deploy has: versioning, rollout, rollback, caching, provenance. You also inherit something ordinary application deploys usually avoid. A run can keep executing an artifact after that artifact has been found unsafe.

That is where the design gets interesting.

The deploy needs a control plane

Once skills are shared by many agents, publishing and governance should be separate from the runtime path that resolves and loads them.

The registry owns the authoritative version records. Skill packages live in immutable artifact storage. The control plane publishes compact catalog snapshots that resolvers can keep regionally or locally.

The runtime should not query the registry database every time an agent considers a skill. Normal resolution should continue during a temporary registry outage using the latest valid catalog snapshot.

Revocation needs a different path. A dangerous executable cannot wait for the next ordinary catalog refresh, so the loader needs a faster view of revocation state.

Architecture diagram showing a production Agent Skills platform with an agent runtime, skill resolver, regional catalog snapshot, eligibility and policy filtering, compatibility checks, package loader, local package cache, and context builder. A separate control plane manages the skill registry, immutable versions, dependency metadata, evaluation status, rollout configuration, artifact storage, and a fast revocation path for privileged skill content.

Production skill infrastructure separates publishing and governance from the high volume runtime resolution path. Immutable packages and regional catalog snapshots keep normal resolution local, while revocation has a faster path into privileged execution.

Open diagram

Publishing, ownership changes, evaluation status, and rollout updates are relatively low volume. Skill resolution and loading happen continuously while agents run. The two paths should scale independently.

Publishing a skill is a deployment flow

A production skill should not move from a developer directory directly into an agent run.

I would make the publishing path explicit.

Developer change
      |
      v
Build immutable package
      |
      v
Compute content hash
      |
      v
Attach provenance and signature
      |
      v
Static validation
      |
      v
Dependency validation
      |
      v
Offline evaluation
      |
      v
Publish as CANDIDATE
      |
      v
Shadow resolution
      |
      v
Canary
      |
      v
ACTIVE
      |
      v
Regional catalog snapshots

The package entering this flow should already have an immutable content identity. Validation and evaluation operate against those exact bytes.

Once the version becomes active, the resolver should be able to tell which deployment attempt made it active. The same candidate version may be rolled out, rolled back, fixed operationally, and attempted again later.

I would keep that rollout state separately.

SkillDeployment {
    skill_id

    baseline_version
    candidate_version

    rollout_generation
    allocation

    eval_status

    catalog_snapshot_version

    status
}

rollout_generation distinguishes separate deployment attempts of the same version.

That becomes useful when someone asks why version 18 was selected on Tuesday but not during another version 18 canary on Friday.

A version is a deploy, so pin it

Version 17 must mean exactly one set of bytes.

skill_id      = payment_incident_investigation
version       = 17
content_hash  = sha256:9b2...

Edit the instructions, replace a script, touch a template. That is version 18, or it is nothing. A published version is never mutated in place.

This sounds like hygiene. It is load bearing. Caching, debugging, rollout, rollback, and revocation all depend on the same version resolving to the same content.

Give versions a lifecycle in the registry.

DRAFT
CANDIDATE
ACTIVE
DEPRECATED
REVOKED

ACTIVE means new runs may resolve to it.

DEPRECATED means new runs should move away from it while existing runs may continue according to policy.

REVOKED means the version is no longer trusted.

Now consider the timeline.

10:00   R9831 starts
        payment_incident_investigation v17 selected

10:05   v18 becomes ACTIVE

10:12   R9831 reaches the next investigation step

At 10:12, R9831 should still use version 17.

Changing procedure in the middle of one logical run produces execution that is difficult to reproduce. Earlier decisions were made with one set of instructions while later decisions use another.

Persist the selected version before behavior derived from that skill begins.

SkillManifest {
    run_id          = R9831
    skill_id        = payment_incident_investigation
    version         = 17
    content_hash    = sha256:9b2...

    activation_reason
    loaded_sections[]
    scripts_allowed[]
    resolved_dependencies[]
    trust_decision
}

That ordering matters.

If the resolver selects version 18, executes skill behavior, crashes, and only then attempts to persist the manifest, recovery may resolve version 17 instead. The run has now used two procedures without an authoritative record of the transition.

Selection should become durable before skill driven execution starts.

A missing pinned artifact should also never cause implicit fallback. If R9831 is pinned to version 17 and version 17 cannot be loaded, silently giving the run version 18 destroys the property that pinning was meant to provide.

Not every skill needs run level pinning. A read only reference skill may safely refresh. A procedural skill that controls how work gets done usually should not. Put that behavior in skill metadata.

New versions need controlled rollout

Version 18 should not replace version 17 for every run at once.

Static validation checks package structure and metadata. Dependency validation confirms that required tools, schemas, runtimes, and other skills exist. Offline evaluation checks known tasks.

Shadow resolution runs the new resolver decision without changing production behavior. It answers a useful question: if version 18 were eligible now, which tasks would select it instead of version 17?

Then start a canary.

baseline_version  = 17
candidate_version = 18

allocation = 5 percent

The resolver assigns a version to an eligible new run and persists that choice into the SkillManifest.

Changing the rollout percentage changes new assignments. It does not rewrite versions already pinned to active runs.

Rollout controls which version new runs receive.

Pinning controls what running work keeps.

Neither is revocation.

Revocation is not rollback

Suppose version 18 turns out worse than version 17. It uses more steps per task and causes more human corrections.

That is a rollback.

New runs move back to version 17. A run already pinned to version 18 may be allowed to finish. Version 18 is not unsafe. It is simply worse.

Now suppose version 17's query script is discovered to exfiltrate credentials.

10:00   R9831 starts on v17

10:11   v17 marked REVOKED

10:12   R9831 reaches the next investigation step

Normal pinning says R9831 stays on 17.

That rule cannot apply unchanged to unsafe executable content.

Revocation overrides the pin.

The response does not have to be identical for every content type.

Content class      Under REVOKED
---------------    ---------------------------------
INSTRUCTION        may continue, flagged, policy decides
REFERENCE          may continue, flagged, policy decides
EXECUTABLE         blocked
TOOL_BINDING       blocked

At 10:12, R9831 asks to load scripts/query_incidents.py. The loader checks the current revocation state, sees that version 17 is revoked, and refuses privileged execution.

step_id           = 14
requested         = scripts/query_incidents.py
skill_version     = 17
revocation_state  = REVOKED
decision          = BLOCKED

Policy can then fail the run, degrade it to instructions only, require human intervention, or explicitly migrate it to another version and record the discontinuity.

Silently continuing is not one of the options.

Caches make this easy to get wrong. The package cache should still contain version 17. Immutable content is exactly what makes package caching useful.

A cache hit is not permission to execute.

The cache identifies the bytes. Revocation decides whether those bytes may still be used for privileged behavior.

Normal catalog updates can propagate on a slower cadence. Revocation needs a faster path because the stale period after a dangerous version has been identified is the period that matters most.

Rollback protects new runs from a bad version.

Revocation protects every run from a dangerous one.

Side by side timeline comparing rollback and security revocation for versioned Agent Skills. In the rollback case, new runs return to version 17 while an existing run pinned to version 18 may continue because the version is still trusted. In the revocation case, run R9831 is pinned to version 17 when a malicious query script is discovered, and the loader blocks executable use of version 17 even though the run selected it earlier.

Rollback changes version selection for new runs while existing runs can remain pinned. Revocation changes whether an artifact is still trusted, so privileged use can be blocked even for runs that selected the version earlier.

Open diagram

The model does not draw the security boundary

The organization has 2,000 registered skills. Nobody loads 2,000 full skill packages into a prompt.

The resolver narrows first.

2,000 registered
    |
    | tenant
    | agent identity
    | environment
    | permissions
    | trust state
    v
  120 visible
    |
    | task intent
    | description
    | activation hints
    v
   12 candidates
    |
    | dependencies
    | compatibility
    | policy
    v
    3 eligible
    |
    | selection
    v
  1 or 2 loaded

The first cut is infrastructure.

A skill that requires deploy.execute does not appear in the candidate set of an agent without that permission. It is not loaded and then controlled by a sentence in the prompt.

The model can help choose among eligible skills. It does not define eligibility.

Selection is also different from loading.

Version 17 may contain:

SKILL.md                 7K tokens
payment_runbook.md      28K tokens
incident_examples.md    18K tokens
schema_reference.md     12K tokens

The first step may need only the main instructions. A later step may load one section of the payment runbook.

The package is the available material. Context is the selected view for the current decision.

Diagram showing progressive Agent Skill resolution from two thousand registered skills to a smaller visible set filtered by tenant, agent identity, environment, permissions, and trust state, then to task matched candidates, compatibility checked eligible skills, and finally one or two selected skills. Only required sections of the selected skill are loaded into model context while other package resources remain available on demand.

Skill discovery, selection, and loading are separate stages. Infrastructure removes skills the agent cannot use before model based selection, and only the required parts of selected skills enter context.

Open diagram

Resolution is a high volume path

The registry should not sit directly on that path.

Suppose there are 10,000 active agents and each performs several skill resolution operations while handling a task. Resolution volume quickly becomes much higher than publishing volume.

Publish compact versioned catalog snapshots regionally. Build indexes over the metadata used for eligibility and candidate matching. Cache immutable packages close to the runtime.

The normal path becomes:

Agent Runtime
      |
      v
Regional Catalog Snapshot
      |
      v
Skill Resolver
      |
      v
Local Package Cache

The authoritative registry can be temporarily unavailable without stopping already published skills from resolving.

The resolver should record the catalog snapshot version used for selection. That becomes useful during partial rollout or when two regions briefly have different catalog generations.

Revocation again has a stronger freshness requirement. A resolver using an older catalog may still nominate a revoked version. The loader should check a fresh enough revocation source before privileged content is executed.

Trust is per file, not per package

Version 17 ships instructions, a runbook, two scripts, and a tool declaration.

They arrived together. They do not need to receive the same trust.

INSTRUCTION      review

REFERENCE        review

EXECUTABLE       review + signing + sandbox + approved publisher

TOOL_BINDING     separate policy because it changes what the agent can reach

A skill imported from another team might be allowed to contribute its runbook while its scripts remain disabled.

skill trusted = true is too broad to be useful.

For every immutable version, record provenance such as publisher, source repository, source commit, content hash, build identity, and signature.

When R9831 runs a script, the platform should know which exact bytes were approved and where they came from.

Dependencies are another part of trust and compatibility.

Version 18 may require observability_mcp >= 12, incident_schema = 7, and production_log_analysis_skill >= 5.

Resolve those requirements before the skill becomes active for the run.

Reject cycles, missing required dependencies, and incompatible versions before loading instructions that assume those dependencies exist.

An agent should not reach step 14 and discover that the procedure it has been following depends on a tool schema unavailable in its environment.

When the skill layer fails

Several failures are specific to this design.

A regional catalog is stale after revocation. The resolver may still select version 17. A fresh revocation check at privileged load time prevents that stale catalog from becoming permission to execute.

A pinned dependency disappears. R9831 is pinned to skill version 17, but the required observability service version is removed. Pinning the skill does not automatically pin every remote dependency. The skill metadata should say whether dependencies are pinned, re resolved within a compatible range, or cause the run to stop when unavailable.

The resolver chooses a version and crashes before recording it. Persist the SkillManifest before executing behavior derived from the selection. Otherwise recovery can choose another version and silently change the procedure.

The package store is unavailable. If the exact pinned package exists in a verified local cache, use it. If it does not, fail the load. Do not substitute another version.

Revocation races with execution. A loader can check version 17, receive an allowed result, and then see the version revoked immediately before starting the script. For higher risk executable content, the execution boundary may need a revocation epoch or another short lived authorization so the gap between check and use remains bounded.

Regions resolve from different catalog generations. This can happen during rollout. Record the catalog snapshot version and rollout generation in the run so different selections remain explainable.

These failures do not need one global recovery rule. They need enough identity and state that recovery does not silently change the artifact being executed.

What this design trades away

Version pinning gives reproducibility, but it delays improvements. A six hour run pinned to version 17 does not automatically receive a bug fix in version 18. That is intentional for procedural consistency. Security revocation is the escape hatch when continuing the old version is more dangerous than changing procedure.

Progressive loading saves context but can add runtime fetches. Loading every reference eagerly spends tokens and can bury useful evidence. Loading on demand adds latency and another dependency during execution. Different skill types will choose different points on that curve.

Immutable versions increase artifact retention. In return, they make debugging, caching, rollback, and provenance much simpler. Content addressed storage and deduplication reduce the storage cost, but version retention still needs policy.

Fast revocation improves containment but can reduce availability. If the revocation path is unavailable, privileged executable content may need to fail closed while read only reference material continues under a bounded stale policy.

Regional catalog snapshots remove the central registry from the hot path. The cost is that ordinary metadata becomes eventually distributed. That is acceptable for most rollout changes because every resolution records the catalog version it used. Security revocation gets a separate faster path because it has a different correctness requirement.

These are deliberate tradeoffs. The same policy does not need to apply to a read only reference skill and a production deployment skill containing executable code.

The invariants

  1. A skill version has immutable content identity. It is never mutated in place.

  2. Publishing and rollout happen in a control plane. Normal resolution uses locally available versioned catalog state.

  3. A run persists its selected procedural skill version before executing behavior derived from that skill.

  4. A missing pinned artifact never causes implicit fallback to another version.

  5. Discovery is filtered by infrastructure before the model selects. Loading is separate from both.

  6. Instructions, references, executables, and tool bindings can carry different trust.

  7. Dependencies resolve before activation, not halfway through the procedure.

  8. The SkillManifest records the selected skill version, content identity, catalog generation, loaded sections, resolved dependencies, and trust decisions.

  9. New versions pass validation, evaluation, and controlled rollout before broad production use.

  10. Rollout and run level pinning solve different problems.

  11. Rollback and revocation are different operations.

  12. Revocation overrides normal pinning for executable and privileged behavior.

  13. A package cache hit never bypasses current revocation policy.

  14. Temporary control plane failure does not stop resolution of already published and locally known safe skills.

  15. Every skill influenced model decision can be traced back to the exact immutable package, deployment generation, and version that supplied it.

A skill is a deploy. Most of the work is making that true. The part that matters most is knowing which of your undo buttons is safe to press while something is still running.