One key, many models: how to choose an aggregating relay service

Why aggregate at all

Say your product uses Claude for long-document analysis, Gemini for multimodal work and DeepSeek for high-volume lightweight tasks. Talking to three vendors directly means:

• three API keys and three rotation schemes
• three sets of error codes and retry policies
• three bills and three ways of accounting for cost
• three rate-limit adaptations

An aggregating relay solves all of that with one key and one base_url — provided you pick the right one.

Six things to check

1. Compatibility: is it standard OpenAI format

Almost every client and SDK supports a custom base_url and the OpenAI-compatible format. If an aggregator requires its own SDK or a proprietary protocol, walk away — your code becomes locked to it, and migrating later is expensive.

2. The model coverage has to be checkable

Do not go by the advertised model count ("500+ models supported"). Pull the actual list from its model listing endpoint and check whether the specific models you need are in it.

The service detail pages on this site show the model list our probe actually retrieved.

You can compare directly which services really carry the models you need.

3. Is the failover real

One of the core values of an aggregator is switching to a backup route automatically when an upstream fails. How to check:

• Ask which upstreams their Claude traffic goes through. If they cannot say, or are vague, there is no real multi-upstream setup.
• Watch how they behave during a large-scale vendor outage. You cannot trigger this on demand, but user reports are informative.

4. Is the billing basis consistent

This is where things most often go wrong. Confirm:

• whether the rate is a flat discount across all vendors or priced separately per model family
• whether cache-hit discounts are passed through (the vendor charges 10% on cache hits — does the aggregation layer give you that)
• whether input and output are priced separately

Some aggregators advertise a "flat rate" as a selling point while charging a high rate on expensive models and a low rate on cheap ones. It looks uniform and actually saves you nothing.

5. Vendor lock-in risk

Using an aggregator means adding another layer in the middle. Assess:

• can you export your call logs and usage data
• how expensive is migrating to a direct connection (if you used standard OpenAI format, changing base_url is the whole job)

Recommendation: keep base_url and the key in configuration, never hard-coded. Then switching providers is an environment variable change.

6. Sanity-check the pricing

Use our official model price list as the baseline:

• For Chinese models, a rate below 0.5 is plausible (resold credit plus volume discount)
• For flagship Claude or GPT models, a rate below 0.2 needs an explanation
• A rate above 1.0 means they are reselling vendor credit at a markup — worth it only if they add something you value

When not to use an aggregator

• You only use one family of models: connect directly, or to a service specialising in that family. One less middleman, one less risk.
• You are extremely latency-sensitive: every extra hop is another network path.
• You have strict compliance requirements: data flow has to be explainable, and a middle layer makes audits harder.

When an aggregator is the sensible choice

• Your product needs several model families and you need to switch quickly (say, A/B testing on cost or quality)
• A small team has no capacity to maintain several integrations
• You need CNY payment and a domestic network path

In one line: an aggregator trades "one more middleman" for "a pile less integration work". When your integration cost exceeds that risk, it pays for itself.