Cached vs uncached tokens: what changes in cost and usage

Separate reusable context from changing context

System instructions, large reference documents, tool definitions and stable examples can be repeated across many requests, while the user question, recent conversation state and retrieved evidence may change each time. Measure those components separately. A workload with a large stable prefix can behave very differently from one where almost every token changes on every request.

Compare provider-reported usage and price

Record uncached input, cached input and output tokens from representative successful requests. Apply the current provider rates to each category and calculate cost per completed task. Do not use an old pricing table as a permanent assumption because cache pricing, minimum cache sizes and retention rules can change.

Use caching and reduction together carefully

Caching can reduce the cost of stable repeated input while token optimisation can reduce avoidable dynamic context. They are complementary when the request design preserves cacheable prefixes and removes only content that is no longer needed. Validate both quality and provider-reported usage before changing production traffic.

Measurement checklist

1. Choose a representative completed task, not an artificial one-line prompt.
2. Record the selected model, provider input, cached input, output, retries and final result.
3. Change one optimization mechanism at a time so the cause remains visible.
4. Verify required identifiers, tool calls, code changes or business fields.
5. Keep passthrough available when the reduced request does not pass.

Varion commercial evaluation

Varion keeps proprietary product implementation details private. Evaluate Token Optimisation on representative traffic and judge it by the measured commercial result. New verified users receive 100,000 processed input tokens and 50 local test runs.