AI library

Practices

Cost and caching

Last reviewed 15 October 2026

Overview

LLM costs are driven by tokens: input, output (including thinking), and cached input. Most savings come from not reprocessing the same context, choosing the right effort and model for each task, and batching work that is not urgent.

Key concepts

  • Price per million tokens: input and output priced separately; output costs more.
  • Prompt caching: stable prefixes (tools, system prompt, documents, history) are reused at a much lower read price. Requires an exact prefix match.
  • Batch processing: asynchronous jobs at about 50% off.
  • Effort: lower effort means fewer thinking tokens and tool calls.
  • Model tier: smaller models cost less, but may need more retries.
  • Cost per completed task: the real metric. A cheaper request that fails is not cheaper.
  • Context hygiene: trimming large tool results and stale history reduces every later request.

How to use it

Free wins first, then trade-offs:

  1. Turn on prompt caching; keep timestamps and per-request data out of the cached prefix.
  2. Batch non-urgent work.
  3. Trim tool outputs and drop stale context.
  4. Lower effort where quality holds (chat, classification, subagents).
  5. Only then consider cheaper models or multi-model cascades, and measure them with evals.

Other levers:

  • Time-of-day pricing: some providers charge less outside peak hours. DeepSeek's off-peak rates are half its peak rates, so scheduling flexible work matters.
  • Long-context surcharges and introductory prices: check where price tiers change. OpenAI charges more above 272K input tokens, Google's Gemini 4 Argon prices double after 2026-11-30, and Gemini 3.8 Flash prices double on 2027-01-01. Google's Pro models also charge more above 200K input tokens. Claude Haiku 5.5 costs 5x more above 100K prompt tokens, while other current Claude models charge one rate across their 1M window.
  • Cache read discounts differ by model: most providers charge about 10% of input for cache reads, but Claude Opus 5.5 and Sonnet 5.5 charge 5% and Fable 5.1 2.5% (as of 2026-10-08). The deeper the discount, the more a stable prefix is worth. Not every model offers explicit caching: Gemini 4 Argon supports only implicit (automatic) caching (as of 2026-10-15).
  • Cache lifetime matters for idle sessions: once a cache expires, the next request pays full input price for the whole context. Claude Code (2.1.299, 2026-10-14) compacts conversations over 200K tokens after about 54 idle minutes, just before the one-hour cache expires, trading one compaction for a full uncached re-read.
  • Classification-only endpoints: for routing and yes/no checks, OpenAI's Decisions API charges only for input ($0.10 per 1M tokens on gpt-6-luna, as of 2026-10-07) and returns probabilities instead of generated text.
  • Paying for speed: the same model can be bought at higher speed for a higher price: Claude Opus 5.5 fast mode (2x price) and OpenAI's ultrafast service tier on GPT-6 Astra and GPT-6.1 Sol. Use them only where latency matters to users.

Reference prices

Per 1M tokens, input / cached input (cache read) / output, standard paid tier. Checked on the date shown; prices change often, so recheck before relying on them.

Model Price Notes Checked
Claude Fable 5.1 $10 / $0.25 / $50 2026-10-08
Claude Opus 5.5 $4 / $0.20 / $20 Fast mode $8 / $40 2026-10-08
Claude Sonnet 5.5 $2 / $0.10 / $10 Cache reads cut from $0.20 around 2026-10-07 2026-10-08
Claude Haiku 5.5 $0.10 / $0.01 / $0.50 Prompts ≤100K; 5x above. Batch halves both tiers 2026-10-08
GPT-6 Astra $10 / $1.00 / $50 Above 272K input: input 2x, output 1.5x (all GPT-6) 2026-10-06
GPT-6.1 Sol $2 / $0.10 / $10 2026-10-06
GPT-6 Luna $0.10 / $0.01 / $0.50 Decisions API: $0.10 input only 2026-10-07
Gemini 4 Argon $2 / $0.10 / $10 Introductory; $4 / $20 after 2026-11-30. Cached input 95% off. 1M shared input-output limit 2026-10-14
Gemini 3.1 Pro Preview $2.00 / $0.20 / $12.00 Above 200K: $4.00 / $0.40 / $18.00 2026-10-07
Gemini 2.5 Pro $1.25 / $0.125 / $10.00 Above 200K: $2.50 / $0.25 / $15.00 2026-10-07
Gemini 3.8 Flash $0.75 / $0.075 / $3.75 Doubles to $1.50 / $0.15 / $7.50 on 2027-01-01; cache storage $0.50 per 1M per hour (rising to $1.00) 2026-10-06
Gemini 3.5 Flash $1.50 / $0.15 / $9.00 2026-10-07
Gemini 2.5 Flash $0.30 / $0.03 / $2.50 Audio input costs more 2026-10-07
Gemini 2.5 Flash-Lite $0.10 / $0.01 / $0.40 Audio input costs more 2026-10-07
DeepSeek deepseek-flash (V4.1-Flash) $0.30 / $0.006 / $1.20 Peak rate; off-peak is half 2026-10-05
DeepSeek deepseek-v4-pro $1.32 / $0.044 / $3.96 Peak rate; off-peak is half 2026-10-05

DeepSeek peak hours are 01:00–04:00 and 06:00–10:00 UTC on weekdays. Haiku 4.5, now legacy, costs $1 / $5.

Current state

  • 2026-10-15: The Gemini 4 Argon model page lists implicit caching only, with no explicit caching. The 95% cached-input discount above applies to implicit cache hits.
  • 2026-10-14: Claude Code 2.1.299 added idle compaction ahead of prompt-cache expiry; 2.1.300 reports 5-minute and 1-hour cache writes separately in OpenTelemetry.
  • 2026-10-13: Gemini 4 Argon reached the paid Gemini API at $2 / $10 per 1M tokens until 2026-11-30, then $4 / $20, the same as Claude Opus 5.5. Its launch price equals GPT-6.1 Sol.
  • 2026-10-08: OpenAI extended the Ultrafast service tier to GPT-6.1 Sol. Prices are on a separate Ultrafast pricing table, not checked in this library yet.
  • 2026-10-07: Claude Haiku 5.5 launched with prompt-length pricing (table above), and Sonnet 5.5 cache reads halved to 0.05x input. Simon Willison dates the cut to the Haiku launch; the docs give no date.
  • 2026-10-06: Gemini 3.8 Flash introductory prices run until 2026-12-31.

Open questions

  • Prices for Gemini 3.5 Flash-Lite and 3.6 Flash. Partly answered on 2026-10-07: the pricing page lists Pro prices (3.1 Pro Preview, 2.5 Pro) and 2.5 Flash-Lite, but has no 3.x Flash-Lite price.
  • OpenAI Ultrafast prices for GPT-6 Astra and GPT-6.1 Sol: how large is the premium over Standard?

Sources