A repeated request returns with no cached input. Before rearranging your prompt, check whether the first request could establish a reusable prefix, whether the second preserved it, and what the response actually reports. GPT-6 Astra prompt caching is an input-processing mechanism, not a guarantee that similar-looking questions will cost less.
This guide provides a diagnostic process based on official documentation. It does not report a measured cache-hit rate, guarantee a saving or imply that an API account was tested.
Quick read
Key takeaways
- Inspect the actual request, not only its final user message.
- Cache reuse is not the reuse of a finished answer.
- Improve efficiency without keeping obsolete instructions.
Triage a prompt-cache miss
Do not label a case “fixed” only because one later call appears faster. Network conditions, queueing, output length and tool execution can also affect elapsed time.
| Check | Question to answer | Useful next action |
|---|---|---|
| Request identity | Did both calls use the intended configuration? | Compare recorded versions |
| Stable material | Where does the first unexpected difference appear? | Separate fixed and changing inputs |
| Tool definitions | Did names, descriptions or schemas change? | Version the tool set |
| History | Was earlier conversation content rewritten? | Track history transformations |
| Timing and retention | Was reuse still eligible under the documented policy? | Check current model-specific guidance |
| Measurement | Are you inspecting actual cache usage? | Compare usage and diagnostic records |
Read the usage record before changing the prompt
Capture a safe diagnostic record that identifies the request template, model, tool-set version and conversation-history version. Avoid putting secrets or private user content into unrestricted logs. A hash or controlled comparison can identify change without exposing every value.
Inspect usage.input_tokens_details.cached_tokens together with cache_write_tokens and total input_tokens. Cached tokens indicate reused input; cache-write tokens represent a different billing component. A zero cached-token count on an initial request is not by itself a defect: compare it with a later eligible request before diagnosing a miss. Never infer cache reuse solely from a quick response.
Then compare a known request with the next request you expected to benefit from it. Locate the first unexpected difference. Do not begin by moving every instruction around, because that creates a new experiment without explaining the original failure.
Common application-level suspects include a changing timestamp near the beginning, a tool list assembled in a different order, a rewritten summary or a modified instruction block. These are things to inspect, not a claim that each one caused a miss in your account.
Why a matching question is not a matching prefix
The official prompt-caching guide describes reuse of intermediate key-value state for a matching prompt prefix. It is not an answer cache: a later request still has new input to process and a response to generate.
That explains why two different questions can potentially benefit from a shared beginning, while two nearly identical questions may fail to reuse the part you expected. What comes before the changing material matters.
For diagnosis, think of the request as a versioned document assembled by your application. The instructions, tools and history belong in that document too. Looking only at the last user message hides much of the relevant evidence.
Run a controlled prefix experiment
A controlled investigation starts with a baseline request and a narrowly defined variation. Keep the task, output requirements and tool availability stable. Record whether the intended prefix is the same before interpreting usage.
Next, test one suspected source of change. If a dynamic field is unnecessary in the stable section, move it only after confirming that doing so preserves the task's meaning. Do not remove relevant context just to improve a cache metric.
Repeat the check with enough examples to distinguish a reproducible pattern from an isolated observation. Until that experiment has been run, describe the change as a proposed fix.
This approach also makes rollback easier. If output quality declines, you can identify the exact request-layout change instead of untangling several simultaneous optimizations.
Use a three-request worksheet: A establishes the baseline, B repeats the same stable material with a new item, and C changes just one suspected field. Record the request versions, time interval, cached-token count and output acceptance for each. This is an experiment design, not sample API output; there are no invented hit rates below.
Imagine a team evaluating proposed item descriptions against a fixed style guide and schema. The guide and schema are stable; the individual item data changes. That is a sensible situation in which to investigate reusable context.
Now imagine the application inserts a new run identifier before the guide on every call. The team should inspect how the final request is assembled before blaming the model. It should also verify that any reorganization keeps the item data clearly separate from instructions.
For an item-description brief, browse RPG games and write your own notes about how inventory labels communicate purpose and constraints. Those notes can inform a style guide; do not copy game text or present the linked games as examples of Astra caching.
| Request | Keep fixed | Change | Record |
|---|---|---|---|
| A: baseline | Model, tool definitions, style guide and schema | Initial item | Input usage and accepted output |
| B: repeat candidate | The same prefix and configuration | Only the item after stable material | Cached input and output correctness |
| C: suspected cause | Everything except the selected variable | One field or ordering change | Where reuse and quality differ |
Reasoning changes deserve a separate check
The reasoning documentation describes a GPT-6 Astra configuration-update mechanism for changing reasoning effort between responses while retaining the original prompt prefix. Its support conditions include standard single-agent mode.
Do not assume that rewriting configuration anywhere in a request has the same effect. Follow the documented mechanism for the integration in use, and inspect the resulting behavior.
This is particularly relevant when a conversation alternates between difficult analysis and routine follow-up. The objective is not to keep every setting frozen forever, but to change it deliberately without accidentally rebuilding unrelated context.
Do not confuse cache savings with total task cost
The GPT-6 Astra model page separates input, cached input and output pricing. Current pricing must be checked at the time of implementation; this article deliberately avoids promising a fixed percentage reduction.
A useful review looks at the accepted task, not just a discounted component. If an optimization encourages longer outputs, adds retries or breaks the response format, the final workflow may be worse even when part of its input is reused.
Report cache behavior and task quality side by side. At minimum, keep the acceptance result, elapsed time and reported usage together. A dashboard that shows only a rising hit rate can hide a deteriorating user experience.
A style-guide evaluation also needs a player-facing quality check: do the accepted labels help someone understand an action? Explore browser games on Elseland AI for interaction references, then write your own test cases. The collection is a place to play, not evidence of cache performance or model provenance.
Preserve correctness before optimizing reuse
A cache-friendly request is not necessarily a good request. If a tool definition changes because the application changed, keep it accurate. If a user corrects a requirement, preserve the correction. Reusing obsolete context is not a useful optimization.
Set an acceptance gate before making a request-layout change: required fields must remain present, the output must match the current task and no stale instructions may override the latest requirement.
Retain a request-layout change only when the observed efficiency improvement survives the acceptance checks. If the cache metric improves but the output follows obsolete instructions, roll back the change. The objective is a correct result with less repeated input work, not a high hit rate at any cost.
Sources and further reading
- official prompt-caching guide
Cache behavior and usage fields checked September 14, 2026. No measured hit rate or savings are claimed.
- GPT-6 Astra model page
Pricing categories only; check current rates for implementation. This article contains no price forecast.
- reasoning documentation
Support conditions for reasoning configuration updates; not a guarantee that arbitrary request changes preserve reuse.
Next step
