Team budget control: from discovering the overspend at month end to live limits

The typical symptoms

"How much did the API cost this month?"

• "No idea, we will see the bill at month end."
• "One colleague's key blew through its limit and everyone else's work stopped."
• "Last month's bill suddenly tripled and nobody can work out where from."

All three symptoms have the same root cause: usage is not broken down to the level of who did what.

And every control measure — limits, alerts, optimisation — is built on top of that breakdown. Without it, none of the rest is possible.

Step one: split the keys

One key per purpose. Choose the split dimension to match how your organisation works. Two are common.

By person: suits teams where everyone works independently.

key-alice / key-bob / key-carol / key-ci     <- CI gets its own

By project: suits teams where several people work on one product.

key-prod-api / key-staging / key-batch-job / key-internal-tool

When to do both: split by project at the first level, then by person inside each project, if the site supports sub-keys.

Why CI and scheduled jobs must have their own key: their usage pattern is nothing like human calls. They may fire thousands of requests in one run. Mixed into the same key, they make your usage curve unreadable.

Step two: settle the measurement basis

Before building any statistics, answer three questions:

1. Measured in what? Request count, token count, or money?

→ Use money, because that is what you are actually managing. Token counts are an intermediate quantity.

1. Are input and output tracked separately?

→ They must be. Output unit price is typically three to five times input. Tracking only total tokens leads you to misjudge the cost structure.

1. Is cache hits counted separately?

→ If caching is on, the cost of a hit can be as low as 10%. Without separate tracking you will overestimate.

Suggested fields to store: timestamp / key identifier / purpose / model / input tokens / output tokens / cache read tokens / amount.

With those eight fields, any dimension of analysis becomes possible.

Step three: alerts and limits

Alerts, the soft limits

Three thresholds are enough:

The second is the most valuable one: notify at 70% and you still have time to intervene; notify at 100% and it is already too late.

Hard limits

If the site supports per-key spending caps, set them. It is the only measure that genuinely contains a runaway.

Set the cap at expected monthly usage times 1.5. Too tight and you damage normal work; too loose and it means nothing.

If the site has no hard limits, you have two options:

1. Add counting and blocking in your own code: route every request through one entry point and reject once the running total passes the cap.
2. Move to a site that supports limits.

The second is less work, but if your workload is already running, the first is the only answer. Option 1 is easiest in a self-hosted gateway setup; see our article on the hidden costs of running your own relay.

Step four: fold it into a regular review

The point of control is not to restrict usage, it is to make input and output visible. So a regular review is needed, weekly rather than monthly, because monthly lags too far behind.

Each week, look at three numbers:

1. Total versus last week, where within plus or minus 20% counts as normal variation.
2. The top three consumers. If a key suddenly appears on the list, go and find out what it is.
3. Cost per unit of output. If a workload has an output metric, such as how many items were generated or how many documents processed, work out the cost per unit.

The third is the most valuable, because it answers a business question: was this money well spent? The first two only answer "how much was spent".

The three most common runaway scenarios

One: loops

An agent or an automatic retry path written incorrectly, so the same request gets sent over and over.

Signature: usage forms a steep spike within minutes, concentrated on one key.

Prevention: every automatic retry needs a maximum attempt count plus exponential backoff. A bare "retry on failure" is the most common source of runaway cost.

Two: context accumulation

An agent resends the full history every turn. As the number of turns grows, input tokens grow quadratically.

Signature: the input token count of individual requests keeps climbing.

Prevention: trim the history with a sliding window, or use caching. See our article on cache billing.

Three: full reruns during debugging

During development, the whole batch is rerun to verify something. If the test environment uses a production key, you are billed for it.

Prevention: give the test environment its own key, with a small cap on it. Running it dry is the signal that testing should converge.

A low-cost starting plan

If you have nothing in place today, do these three in order:

1. Split the keys — half an hour, and usage immediately becomes distinguishable.
2. Record usage once a day — a script that takes five minutes to write. Three days later you have a baseline and can spot anomalies.
3. Set one alert: notify when usage exceeds twice the trailing 7-day average. That single rule stops most runaways.

Together under a day of work, and it solves the fundamental "we only find out at month end" problem. Further optimisation can come later.

The order matters: visibility first, optimisation second. Optimising while you cannot see anything is the same as tuning parameters blindfolded.