Build consistent prefixes
Keep system instructions, tools and reusable examples stable. Avoid random ordering and unnecessary changes at the beginning of the prompt. Use supported cache keys or retention options only as documented by OpenAI.
Read the usage breakdown
Separate total input from cached input and compare actual cost. Cached tokens can still appear inside total input usage, so a dashboard must not treat them as removed. Inspect provider usage fields for every benchmark.
Local token estimate
This is an approximate comparison, not provider billing data.
Combine caching and reduction carefully
Preserve required stable content for caching, then reduce repeated dynamic history and irrelevant schemas where quality checks allow. Varion’s reporting distinguishes provider-bound token reduction from exact-cache hits, so the two effects are not mixed.
Measurement checklist
- Choose a representative completed task, not an artificial one-line prompt.
- Record the selected model, provider input, cached input, output, retries and final result.
- Change one optimization mechanism at a time so the cause remains visible.
- Verify required identifiers, tool calls, code changes or business fields.
- Keep passthrough available when the reduced request does not pass.
How Varion fits
Varion Token Engine is a gateway and testing platform for reducing eligible input-token waste across supported AI traffic. It reports original and provider-bound input, keeps provider charges separate, and does not claim that every request can be reduced. New verified users receive 100,000 processed input tokens and 50 local test runs.
Frequently asked questions
Are all OpenAI prompts cached?
No. Eligibility depends on current model and platform behavior. Consult the official prompt-caching documentation.
Does a cached token equal a removed token?
No. It is provider-reported cached input, not content removed from the request.