How available niche models like Grok and Llama really are on relay sites

Why availability varies so much for niche models

Whether a model can be used on a relay site depends on three conditions holding at once:

1. A legitimate channel exists upstream, an official API or at least a stable proxy.
2. The site is willing to carry it, and carrying it means maintaining the channel and handling failures, while small volume does not recover the cost.
3. The channel is stable, and some models have only a handful of channels with no replacement if supply stops.

Small models, new models and self-hosted open models usually break down at the first or third condition.

Three kinds of niche supply

One: official channels

The most reliable. For models with an official API, such as some cloud-hosted members of the Grok and Llama families, availability matches mainstream models as long as the site is connected to the official channel.

Identifying features: consistent model naming, presence in /v1/models, well-formed error responses.

Two: self-hosted open models

For models like the open releases of Llama and Qwen, a relay site can deploy them itself or buy capacity from a third-party inference cloud such as Together, Groq or SiliconFlow.

The problem with this supply is that quality is uncontrolled:

• Different quantisation. The same Llama may run at fp16, at int8, or even at 4-bit. The quality loss from 4-bit quantisation is visible on complex tasks.
• Limited concurrency. The throughput of a self-hosted instance depends on the site's hardware.
• Many version branches. Llama 3.1, 3.2 and 3.3 differ noticeably in capability, and a site may label them vaguely.

Three: unknown provenance

This is the category to be most careful about: the site offers a model name, and there is no verifiable information about what is actually running behind it.

Typical signs: the model name does not match the official one, the same name produces significantly different response characteristics at different times, and error messages are bespoke.

Four steps to judge whether a model is worth depending on

Step one: check availability

curl -s https://<domain>/v1/models -H "Authorization: Bearer $KEY" \
  | python -c "import sys,json;print([m['id'] for m in json.load(sys.stdin)['data']])"

Confirm it is in the list first. If it is not listed, it is not there. Do not hope for "it might work even though it is not listed".

Step two: test version consistency

Verify with a question only a specific version can answer correctly.

Asking a model "what version are you" is unreliable, since it will guess from context. A better approach is to probe a capability introduced in that version, or to check for a bug that version fixed.

Criterion: if it behaves like the older version, the version lags.

Step three: test response consistency

Run the same prompt several times at different moments, hours or days apart, and compare:

• whether the output style is consistent;
• whether answers to the same factual question are consistent;
• whether the latency profile is consistent.

Poor consistency means the backend is changing. It may be routing between channels, or the channel itself may be being swapped. For applications that need stable behaviour, such as a pipeline depending on a fixed output format, that is a fatal flaw.

Step four: test the quality floor

Prepare five to ten tasks whose correct answers you know, and run them.

For self-hosted supply this step matters especially. Low quantisation precision does not show up as a wrong answer; it shows up as a vague answer with details missing. Only tasks with definite answers will expose it.

When a niche model is a reasonable choice

• You have a clear reason, such as a particular style or capability that matches your need precisely.
• It is not on a critical path, so you can switch back to a mainstream model whenever something goes wrong.
• Volume is small, so even total unavailability costs little.
• You ran the four tests above and they passed.

When to avoid one

• A critical production path. Niche models have unstable channels, and one supply interruption is one outage.
• You need consistent output. Models of unknown provenance or self-hosted ones may behave differently at different times.
• You need long-term reproducibility, as with evaluation benchmarks or data labelling. Those require "the same input always reaches the same model", and a niche model's channel may change at any moment.
• Volume is large. Large volume means your cost is already enough to negotiate a discount, so there is no reason to introduce uncertainty to save a little.

A pragmatic approach

Treat mainstream and niche models separately. Do not evaluate them with the same standard.

• Mainstream models — GPT, Claude, Gemini, DeepSeek, the Qwen family: mature channels, so focus on availability rate, latency and price.
• Niche models: look at availability and consistency first, price second, because the more likely problem is "works today, gone tomorrow" rather than "a bit expensive".

If you are evaluating a site's niche model supply, keep a record of your tests in the dashboard, with time, model name and result, and test again a week later. Only when the two results agree is there reason to put it into your plan.

Our detail pages list the model count detected by probing, measured from /v1/models, and you can use it to compare coverage breadth across sites. Note carefully, though: it means "how many are listed", not "every one of them is stably available". The latter you have to test yourself.