Comparing review platforms for relay services: what apiranking, Ztest and GatewayBench each measure

Why you need a layer of review rather than a leaderboard

Whether a relay site can be trusted has at least three independent dimensions:

1. Is the model real -- when you pay for Sonnet, do you get Sonnet
2. Is the service stable -- uptime, latency, and jitter at peak
3. Is the price honest -- does the advertised rate match the bill you actually get

Any single leaderboard covers only one or two of these. Treating platforms with different criteria as if they were the same thing is the most common mistake in choosing a site.

Five kinds of platform, five kinds of criterion

1. apiranking -- model verification plus channel-level price comparison

Its differentiator is verification: fingerprint questions, model self-description and response characteristics are used to judge whether the model being called matches what is advertised. It also breaks pricing down to the channel level, since the same model can cost different amounts on different channels.

• Strength: it addresses the prior question of whether the model is genuine.
• Blind spot: verification is sampling, not continuous monitoring, and coverage of stability is limited.
• Best for: people with a hard requirement on model authenticity, used as a pre-filter.

2. Ztest -- testing with real requests

The method is to hit real tasks with real requests and look at success rate and response quality, from a cross-validation angle.

• Strength: it does not depend on what the site reports; conclusions come straight from request results.
• Blind spot: the test set is finite, so sample size alone means long-tail scenarios such as multimodal input, function calling and very long context are not covered.
• Best for: checking whether a site works under your own pattern of use.

3. GatewayBench, delivered through Check4U.ai -- verifiable three-axis metrics

It publishes verifiable metrics across credibility, economics and performance, oriented toward compliance auditing.

• Strength: the metric definitions are clear, which suits situations that need an audit trail or an explanation for a team.
• Blind spot: the granularity is platform-level, so fine differences between individual sites are thinly covered.
• Best for: internal vendor approval inside a company.

4. Helpaio -- community reputation

User reviews fill the gaps that technical testing cannot reach.

• Strength: it reflects things measurable metrics miss, such as support responsiveness, how readily refunds are given, and how problems are handled.
• Blind spot: reputation can be managed, and negative reviews cluster after an incident, so there is a lag.
• Best for: a final check for obvious landmines.

5. Lingjian AI Ladder Board -- an entry-level view

Low barrier, aimed at people encountering relay sites for the first time.

• Strength: quick to absorb, good for building an overall impression.
• Blind spot: coarse, and not suitable for a decision once your requirements are defined.
• Best for: the first thing to read when you know nothing about this market.

How to combine them

A workable order:

1. Start with a ladder-board style platform to build a general impression, and shortlist five to eight candidates.
2. Use a verification tool to remove the doubtful ones, leaving three to five.
3. Use a real-request platform to see how they perform on real tasks, alongside our uptime and latency data, leaving two or three.
4. Search reputation channels for negatives, with keywords like the site name plus disappearing, downgrading or refund, and rule out any site with a clear incident record.
5. Test in small amounts yourself. Do not skip this step: every platform's data comes from someone else's scenario, and only your own usage pattern is the final criterion.

One caveat

These platforms have commercial motives of their own. When you see exclusive, authoritative or only claims, go back to the method and ask three questions: where do the samples come from, how many are there, and what dimensions are measured. A ranking that cannot answer all three is of limited value.

We are also a data source, but we do only two things: probe uptime and latency continuously under one consistent methodology, and label a site's own claims separately from our measurements. We have no verification capability, meaning we do not do model fingerprinting, and we publish no composite ranking. Our about page says this plainly, and you should hold our data to the same standard.