Logging and reconciliation: attributing token spend to the project level

Start with what to record

A workable call log needs nine fields:

Two key points:

1. tokens_in and tokens_out must be separate. Output unit price is typically three to five times input, and combining them leads you to misjudge the cost structure entirely.
2. key_id and project must be separate. One project may use several keys and one key may serve several projects. You need both dimensions to trace in either direction.

Should you compute cost yourself? Yes. The site's dashboard gives you a total, while what you need is an amount broken down by dimension. Maintain your own price table mapping model to input and output prices, and convert each call from usage.

Where to write it

The principle: logging must not affect the main path.

• Write asynchronously. Do not write to a database synchronously inside each request, since that adds latency and a logging failure would drag down the business.
• Write to a local queue or file first, then batch load into storage in a separate process.
• Failures must not block. If the log write fails, drop it and count it. A logging problem must never become a business outage.

A cheap starting point: write JSONL files, one per day. grep is enough for investigation and a script handles aggregation. This scales surprisingly far. Do not reach for a database on day one.

Three reconciliation methods

Method one: against the site's bill, for verification

Method: compare your own sum(cost) against the charge shown in the site's dashboard.

Criteria:

• Difference under 5% → normal, given minor differences in token counting.
• Difference 5% to 20% → worth investigating. Some calls may not have been logged, or your price table may be out of date.
• Difference over 20% → something is wrong. Either the site's billing basis differs from your understanding, for instance it prices input and output the same, or there are calls you are not recording.

The value of this one: it is the only method that can reveal a mismatch between the site's billing and its published prices. Without reconciliation you will never know.

Method two: against your own estimate, for prediction

Method: for each business action, estimate its token consumption and compare with reality.

For example: "generating one article costs roughly 2000 input plus 1500 output". If reality is 8000 plus 6000, there is an extra source of consumption, whether context accumulation, retries, or an over-long system prompt.

Criterion: actual more than twice the estimate → go and find out why.

Three common reasons:

1. Context accumulation, where an agent carries the full history every turn and input grows quadratically.
2. Retries, where an automatic retry after failure is billed every time.
3. An over-long system prompt, where tool definitions and role setup take a large share of input.

Method three: against business volume, for plausibility

Method: total cost divided by business volume gives the cost per unit of output.

For example: the cost per article generated, per document processed, per user session.

Criterion: a sudden jump in this number means investigate immediately.

This is the most valuable of the three, because it converts a technical metric into a number the business understands. Telling a team "the API cost this much this month" is abstract, while "the cost per article went from 0.3 to 1.2" is something everyone grasps.

Three rules that surface anomalies

Rule one: a single request's input tokens above a threshold

What it points to: context accumulation, a sudden change in the system prompt, or a large file accidentally included.

What to do: print the details of the offending request rather than only the number, and look at what is in the content.

Rule two: a sharp rise in requests from one key_id in a short window

What it points to: a loop, a runaway retry, or a leaked key.

What to do: look at the request content first. If the content repeats, it is a loop. If the content pattern is unfamiliar, it is a leak.

Rule three: failure rate rising while business volume is flat

What it points to: a supplier-side fault, a configuration change of your own, or upstream rate limiting.

What to do: group by status and look at the failure reasons. Separate 4xx, your problem, from 5xx, theirs.

A pragmatic implementation order

Step 1: converge base_url and keys into one exit point (half a day)
Step 2: add usage logging, JSONL and asynchronous (half a day)
Step 3: maintain a price table mapping model to input/output price (2 hours)
Step 4: write an aggregation script by project and by day (half a day)
Step 5: set one alert, notify when daily usage passes twice the trailing 7-day average (1 hour)

About two days in total. What those two days buy you: you know where the money goes, you can spot anomalies, and you can explain cost to your team.

By comparison, optimising cost while unable to see usage is the same as tuning parameters blindfolded, because you cannot tell whether a change helped.

One last point

We cannot do your reconciliation for you, since you are using your own keys. But two things help:

1. The verification status and measured data on our detail pages, for judging whether a site is worth continuing with.
2. The official price baseline table at /model/lists, for maintaining your own price table, and in particular for judging whether a multiplier is reasonable.

Reconciliation can only ever be done by you. It is not hard. What is hard is starting.