Your AI bill is mostly the same instructions, repeated
Most of what a business AI tool sends to the model is the same every time: long instructions, examples, rules — and in some workflows, the same document several times over. Without caching, you pay full price for all of it, on every call.
Where the money goes
An AI model is billed by the amount of text sent in and the amount sent back. In business automations, most of what goes in doesn’t change: the instructions that explain the task, the output format, the rules about your part numbers and customers. The part that changes — this invoice, this purchase order, this remittance — is often the smaller share.
Some workflows are worse. To read a purchase order accurately and quickly, our order-booking tool sends the same PDF to the model several times at once: one read for the header, more for the lines, in parallel. Each of those calls paid full price for the entire document.
What prompt caching does
Anthropic’s API, like other major providers, can cache the beginning of a request — everything up to a marker you choose — so the next request starting with exactly the same content reuses it instead of processing it again. As of October 2026, reading from the cache is billed at a small fraction of the normal input price (5% on the current Opus and Sonnet models, 10% on others), while writing to it costs a little more than normal (1.25× for a five-minute cache, 2× for one hour) (cost breakdown). Pricing changes; check your provider’s current rates before you plan around them.
Pay once to store the expensive part, then reuse it for pennies.
The results
After adding caching to the order-booking tool, the first real purchase orders through it showed:
| PO | Reads that used the cache | PDF input cost vs. before |
|---|---|---|
| 2 pages | 5 of 5 | about 70% less |
| 1 page | 3 of 3 | about 58% less |
| 1 page | 3 of 3 | about 58% less |
Counting everything sent, total input per purchase order fell by roughly 35% for one page and 55% for two, and the saving grows with page count. The results were identical to before; only the cost changed. The remittance tool now caches its instructions too, so every remittance after the first in a batch reuses them.
Where caching pays off — and where it doesn’t
- Pays off: the same long instructions on every call; the same document read several times; a batch of similar items processed close together.
- Doesn’t: a document sent exactly once. Caching it costs more than not caching. The saving there comes from caching the instructions in front of it instead.
- Doesn’t: short instructions. Below a minimum size, the provider simply doesn’t cache, and doesn’t tell you.
- Doesn’t: a once-a-day job. A five-minute or one-hour cache expires long before the next run.
Questions to ask about your AI tools
- Do we know what each AI feature costs per transaction?
- Which parts of each request are identical every time?
- Is caching on, and do the usage reports show cache reads, not just cache writes?
- Are we using the right model for each job, or the most expensive one for everything?
If your provider has emailed you that your cache rate is low, it’s worth an hour of someone’s time. The technical details — and the mistakes that leave caching switched on but doing nothing — are in the companion piece.
Related: why prompt caching often saves nothing, and booking customer POs with AI.