From list price to your invoice: a worked cost conversion

The scenario

A support-deflection assistant. Input is a fixed system prompt of about 3,000 tokens plus a retrieved knowledge-base passage of about 2,000 tokens. Output averages 500 tokens. Volume is 20,000 calls a month.

We will convert that into a monthly figure, one effect at a time, and show where each assumption could be wrong.

Step 1: start from the vendor's list price

Take a mid-tier model at $3 per million input tokens and $15 per million output tokens.

Everything below is a multiplication on these two numbers, so it is worth confirming them rather than trusting a remembered figure. Our model price list records vendor list prices with the date they were checked.

Step 2: tokens per call

Input tokens: 3,000 (system) + 2,000 (retrieved passage) = 5,000 Output tokens: 500

Twenty thousand calls a month means:

• input: 100,000,000 tokens = 100 units of one million
• output: 10,000,000 tokens = 10 units of one million

Step 3: apply the list price

• input: 100 × $3 = $300
• output: 10 × $15 = $150
• total at list: $450

Note how much of this comes from output. Output is one tenth of the tokens and one third of the cost, because output is priced at five times input. That ratio is the first thing to check when you compare models.

Step 4: apply the cache discount

This is where the worked example stops being arithmetic and starts being a question about your architecture.

The system prompt here is fixed. If the vendor supports prompt caching and the relay passes the discount through, that 3,000-token block may bill at 10% of the standard input price on cache hits.

Assume a 90% hit rate on the system prompt:

• cached portion: 3,000 × 0.9 = 2,700 tokens per call at 10% of $3 = $0.30 per million
• uncached system prompt: 300 tokens per call at $3
• retrieved passage: 2,000 tokens per call at $3

Per million-token equivalents across the month:

• cached: 2,700 × 20,000 = 54,000,000 tokens at $0.30 = $16.20
• uncached system: 300 × 20,000 = 6,000,000 tokens at $3 = $18.00
• passage: 2,000 × 20,000 = 40,000,000 tokens at $3 = $120.00
• output unchanged: $150

Total with caching: $304.20, against $450 without. A 32% reduction, from one architectural fact.

Two caveats. First, this only holds if the relay passes the discount through — many do not, and that is now a line item worth asking about. Second, cache hits require the prefix to be genuinely identical; a timestamp at the top of your system prompt destroys the benefit.

Step 5: apply the relay's rate

Say the service advertises 0.4.

$304.20 × 0.4 = $121.68 per month

This is the number the sales page would show you. Everything else in this article exists to explain why it is not the number you should plan on.

Step 6: add the effects people leave out

Retries. If 5% of calls fail and are retried, that is 5% more token consumption. $121.68 × 1.05 = $127.76.

Tokeniser differences. The same text can be 10–30% more tokens on one vendor than another. If the relay routes across vendors, your effective token count is not what you measured. Take 15% as a working assumption: $146.92.

Model fallback. If 3% of calls degrade to a cheaper model, cost goes down slightly while quality changes. Not modelled here, but worth tracking.

Reconciliation drift. If billing is computed with a rounding or margin you cannot see, this is where a 5–10% discrepancy shows up. Step 6 exists so you know the magnitude when you go looking.

Step 7: reconcile your first real bill against this

The purpose of building the model above is that you can now compare. When the first invoice arrives:

1. Pull your own usage totals from your logs
2. Multiply by the list price and the advertised rate
3. Compare with what was deducted

If the gap is under about 10%, it is explained by retries and tokeniser variation. If it is larger, ask. That question is much easier to ask with the arithmetic already in hand.

What this exercise actually teaches

The headline rate is the least interesting number in the calculation. In this example, the difference between 0.4 and 0.35 is about $15 a month. The difference between having caching and not having it was $146.

Optimise the architecture first, then shop for rates. And when you do shop, put your own numbers into the calculator on our rankings page rather than comparing headline rates between services.

Related reading

• What a relay service's "rate" actually means: 1.0, 0.3 and 0.05 explained (guides)
• 2026 model API pricing review: who actually got cheaper (guides)
• prompt caching on relay services: does it work, and what is it worth (guides)