Prompt Caching Discount Estimator

Why Prompt Caching Matters for AI Developers

If you're running an LLM-powered application at scale, your system prompt is almost certainly your largest and most repetitive cost driver. Every time your application calls the API — for a user question, a document classification, a support ticket response — you're sending the same thousand-token system prompt over and over again. Each repetition is billed at full input token rates.

Prompt caching solves this by letting you store that static prefix on the provider's infrastructure. Subsequent calls that reuse the same prefix are charged at the cache read rate instead: 90% cheaper for Anthropic, 50% cheaper for OpenAI, 75% cheaper for Google. For applications making thousands of daily calls to the same large system prompt, the monthly savings can be substantial.

Anthropic Claude: Explicit Cache Control

Anthropic's implementation requires you to explicitly mark blocks for caching using a cache_control parameter in the messages API. Cache writes cost 25% more than standard input rates (to cover the overhead of writing to cache), but cache reads cost only 10% of the input rate — a 90% discount. The cache persists for at least 5 minutes and resets on each access. For production applications with consistent traffic, the write cost is negligible compared to the read savings.

OpenAI GPT-4o: Automatic Caching

OpenAI caches automatically — there's no API parameter to set. Any prompt prefix longer than 1,024 tokens that appears in recent requests is automatically cached. Cached tokens are billed at 50% of standard input rates. The lack of an explicit write cost means OpenAI caching is always profitable from the very first repeated request. The downside is less control: you can't force cache writes or inspect cache hits programmatically.

Google Gemini: Context Caching with Storage Fees

Google's approach is more explicit than OpenAI but adds a storage dimension: you create a cached content resource with a specified TTL, and pay an hourly storage fee per million cached tokens while it exists. Cache reads are billed at 75% off input rates. For workloads with consistent traffic throughout the day, the storage cost is easily offset by the input savings. For sporadic workloads, evaluate whether storage fees erode the discount.

When to Use Prompt Caching

Caching delivers the highest ROI when your system prompt is long (thousands of tokens), your query volume is high (hundreds or thousands per day), and your prompt content is mostly static. Common use cases include: RAG systems where the same large document context is queried repeatedly, customer support bots with detailed product knowledge bases, code review tools that embed an entire codebase, and multi-step agent workflows that reuse the same tool definitions across many turns.

Caching is less effective for one-off queries, personalised prompts that change per user, or very short system prompts below the provider minimum (typically 1,024 tokens).