LoRA adds a small trainable update to selected base weights. Inference still needs the base model and the selected adapter's weights for the request. Sharing the base is attractive, but a request with a cold adapter may wait for it to load, and mixing many distinct adapters in one batch has engine-specific limits. The S-LoRA paper explores heterogeneous batching and paging of adapter weights. vLLM's current LoRA documentation supports serving and dynamic adapter resolution, with configuration limits for concurrency and rank. These are implementation possibilities, not a guarantee that arbitrary adapter combinations have the same latency as one hot adapter.

I would route on (base model version, adapter ID, adapter version) after authenticating the tenant. Keep hot adapters resident where their traffic is routed, plus a bounded host or local store for warm adapters. Cold adapters go to a loading lane with an explicit first-token budget or a clear asynchronous response. Batch together where the engine can handle heterogeneous adapters profitably, but avoid dispatching every rare adapter to every worker. Measure adapter load time, hit rate, GPU memory per rank and adapter, per-adapter queue age, batch efficiency, and worst tenant latency. The adapter residency policy should have quotas so one tenant's rapid rotation cannot evict everyone else's hot set.

The update is a consistency problem. Store new weights under an immutable version, verify compatibility with the pinned base and tokenizer, load into a canary, and evaluate the tenant's quality and latency slice. New requests can move to the new version after activation. In-flight requests retain the old version until completion or a controlled restart. Do not mutate a buffer under an active batch and call the old and new outputs equivalent. Retire old residency once its requests drain, with a rollback route that still has the old version available.

Security is easy to miss in a performance discussion. The client cannot select another tenant's adapter by spelling its name. The route derives tenant identity from authentication and an authorized mapping, and only reviewed adapter bytes from a trusted store may be loaded. Some serving systems allow runtime loading through administrative endpoints. vLLM warns that its dynamic load feature is unsafe outside an isolated, fully trusted environment. Keep those endpoints behind deployment control, not directly exposed to model output or tenant traffic.

Would I promise one shared pool for every tenant? Only after testing the real skew. A few high volume customers may deserve isolated capacity to protect their tails and support custom releases. The shared pool can handle the long tail if cold-start delay is acceptable. Compare that against the cost of overprovisioning per tenant and the operational cost of many adapter versions. “The adapter is small” does not settle the routing, loading, or versioning decision.