Per request
Cap input, output, retrieved context, tool calls, retries, and wall-clock execution before work starts.
AI cost controls
Bound spend at the request, tenant, and service levels.
A monthly alert reports a cost problem after the system has already accepted the work. Production controls decide which work may consume tokens, tools, and retries before the spend occurs.
The direct answer
How do you keep production AI spend predictable?
Set token and tool budgets before each request, enforce cumulative limits per tenant and service, reserve hard gates for spend the system must not exceed, and evaluate every caching or routing change against the same quality gate as the expensive path.
An alert asks someone to act. A hard gate rejects, queues, degrades, or routes work automatically when the authorised budget is exhausted. Both are useful, but they control different risks.
Per tenant
Track cumulative usage against an explicit allowance so one tenant cannot consume the shared service budget.
Per service
Apply hourly, daily, and monthly gates with a defined response: queue, degrade, deny, or require approval.
Per outcome
Measure cost only alongside task success, evidence quality, safety, latency, and escalation rates.
Control stack
Use prevention before optimisation.
-
Reject unbounded work
Validate requested context, output length, tools, retries, and concurrency against a declared request budget.
-
Attribute every unit
Record input, cached input, output, retrieval, tool, retry, and model-route usage against the request, tenant, feature, and outcome.
-
Separate alerts from gates
Alert on trends that need investigation. Gate the absolute limits that must hold without waiting for a responder.
-
Cache stable work
Reuse stable prompt prefixes and deterministic retrieval results only when cache keys, freshness, access boundaries, and invalidation rules are explicit.
-
Route by measured difficulty
Use a lower-cost model only for cases where the evaluation set shows it meets the same release criteria. Escalate uncertain or high-consequence cases rather than silently lowering quality.
-
Review cost and quality together
Compare candidate configurations on cost per accepted result, not cost per request. A cheaper route that increases retries, correction, or escalation may cost more per completed outcome.
Engineering reasoning: optimisation follows control because caching and routing cannot protect a service whose maximum request and tenant exposure were never bounded.
Public sources
Primary references for caching controls
OpenAI prompt caching
OpenAI documents that eligible requests can reuse exact prompt prefixes and exposes cached-token usage for monitoring. This is specific to OpenAI's current API behaviour and does not prove that caching is safe, effective, or cost-saving for another provider or workload. Read the OpenAI guide (retrieved 2026-08-16).
Anthropic prompt caching
Anthropic documents automatic and explicit cache controls, cache lifetimes, and usage accounting for its API. These are vendor-specific mechanics; they do not define tenant budgets, freshness rules, or the quality gate for the reader's system. Read the Anthropic guide (retrieved 2026-08-16).
Published method
Agent frameworks for this decision
For agent-based workflows, these documents expand on the decision above. They describe our method, not client results.
AI Agent Control Checklists
Use the operational controls to assign cost-threshold breaches to an owner and test the response alongside quality.
Agent Reference Architectures
Choose the least complex execution path that meets the quality gate before paying for additional agent coordination.
Next decision
Review one system against the checklist
Bring the release candidate, its current controls, and the decision you need to make. The first conversation identifies the evidence that exists and the gaps that still need owners.
We review your enquiry and reply by email. No commitment to a paid project; scope and terms are agreed before work begins.
Prefer email? dev@alpha-ai.dev