Model and Inference Engineering · Staff
An image request fits the text token budget. Why does the GPU run out of room?
The question
Interview question
A gateway admits a request with 2,000 text tokens and six high-resolution images. It reserves capacity based only on the text estimate. The request enters a crowded multimodal fleet, misses its deadline and sometimes fails for memory pressure. Product says to resize every image to low detail. What should admission count, and when would that resize make the answer wrong?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
The text counter measured one component of the work. Depending on the model, an image may be resized, split into patches or tiles, encoded by a vision path, and represented by visual tokens or features consumed during prefill. Image dimensions, aspect ratio, detail setting and number of images affect cost differently across models. There may also be temporary preprocessing and activation memory apart from the autoregressive KV cache. We should not translate one provider's billable image tokens directly into a universal GPU memory formula.
I would trace the exact serving path for this model and deployment: original image dimensions, actual post-resize dimensions, patch or crop count, visual tokens entering the language model if exposed, encoder time, prefill time, peak memory, KV reservation and queue wait. OpenAI's image and vision guide documents model and detail dependent image accounting for its API. A self-hosted vision model may follow a different preprocessing and memory path. The gateway needs a versioned estimator for the selected route, with conservative headroom and measurement of estimate error in production.
Admission should consider deadline as well as memory. Six images can fit on a GPU and still occupy enough prefill time to make every short request behind them miss first-token latency. Put bounds on count, resolution and total estimated work. Reserve against the actual model lane and in-flight budget, and release unused reservation when the real preprocessing result is known. Use fair scheduling so an image-heavy tenant cannot consume the interactive pool with ordinary-looking QPS. If the estimate is too uncertain, use a queue with a truthful completion time or reject before expensive partial processing.
Low detail is an option, not a default correctness fix. If the user asks whether a label in the corner of a medicine package reads 10 mg or 100 mg, a resize that removes the digit changes the task. For a broad image caption, low detail may be sufficient. For small text, defects, tables or diagrams, we might crop a relevant region at higher detail, request a better image, or use a separate OCR path. A first low-detail pass can guide a second pass, but account for the extra latency and verify it actually locates the right region. The product must decide whether an answer with lost detail is acceptable, not silently trade away the evidence to protect capacity.
What if the provider hides visual-token counts? Use empirical distributions by model, size and detail class with a safety margin, monitor rejected and misestimated requests, and keep an overload fallback. The admission contract can be approximate without pretending its estimate is exact. Test tall scans, tiny serial numbers, six ordinary photos, one huge image, and mixed text plus tools. Video requests hide behind a normal text QPS chart asks why video request load hides behind a text QPS chart. This question is per-request accounting and quality for images, where a text-only budget is the wrong admission unit.
Continue reading
Related questions
Read beyond the question
Explore more model and inference engineering
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →