Who handles your request: the data boundaries inside a relay chain

The chain, laid out

When you call a relay service, the path your prompt takes is roughly:

your application
  → the relay service's gateway
      → possibly another relay or aggregator
          → the upstream channel (resold credit, resold subscription, reverse-engineered endpoint)
              → the model vendor

Every arrow is a hop where your data exists in somebody's memory or on somebody's disk. Directly calling the vendor removes the middle three.

This is not an argument against relay services. It is an argument for knowing how many hops you have and what each one does with the content.

What actually varies between services

Most relay services will tell you they "do not retain data". That statement is worth very little on its own, because it does not distinguish between:

• Pass-through: the request is forwarded and nothing is written. Logs may still record metadata (model, token counts, timestamp, key).
• Short retention: the full request and response are kept for a period for debugging or dispute resolution.
• Persistent storage: the content is written to a database and kept indefinitely, sometimes to build a dataset.

The distinction that matters most to most businesses is between the first two and the third. Metadata logging is normal and reasonable — you cannot run a metered service without it. Retaining prompt content is a different commitment.

The questions worth asking

Ask these in writing, not in a chat window, so you have the answer on file:

1. Are prompt and response contents retained? If yes, for how long, and where?
2. Which upstreams does traffic go to? Name them, and say whether any are outside the jurisdiction.
3. Is content used to train or fine-tune anything?
4. What is encrypted in transit? HTTPS externally, and what about the hop to the upstream?
5. Can you provide logs showing our usage without prompt content? Useful for reconciling bills without exposing data.
6. What happens to our data if we close the account?

A provider that answers all six clearly is one you can take to a security review. A provider that answers "we take privacy seriously" has answered none of them.

Why low-cost channels deserve extra attention

The channel a relay service buys from changes the data risk:

• Resold vendor credit: data goes to the vendor, as it would if you called the vendor. The relay sits in the middle but the ultimate destination is the same.
• Resold subscription quota: your request runs through somebody's personal subscription account. Data handling is that vendor's, but the account-sharing aspect means your traffic is comingled with strangers'.
• Reverse-engineered endpoints: these often involve third-party infrastructure of unknown provenance, and are the highest risk category.
• Misrepresentation: if the service is secretly routing to an open-source model, that model may be self-hosted on infrastructure with no stated data policy at all.

The last two are the cases where "we do not retain data" is least verifiable and least likely to be accurate.

What the questions look like from your side

If you are filling out your own security review, the three answers your team needs are:

1. Can a third party read the content of our prompts?
2. Does this data leave the jurisdiction, and if so where does it go?
3. If there is a breach, who is liable and what is the recourse?

The relay service is the entity that has to answer all three, including for its upstreams. If it cannot, that is itself the answer.

Reducing exposure without giving up relays

You do not have to choose between using a relay and protecting data. Practical measures:

• Send only what is necessary. A lot of prompts contain identifiers, internal URLs, customer names and file paths that the task does not need.
• Redact before sending. Strip names, account numbers and hostnames in a preprocessing step. This is usually cheap and it removes most of the exposure.
• Separate workloads. Put the sensitive ones on a direct vendor connection and the high-volume, low-sensitivity ones on a relay.
• Keep a written record of the channel. Note which service and which upstream each workload uses, so that when a question comes up in an audit you can answer it without reconstructing anything.

When to skip the relay entirely

If your prompts contain personal data, regulated data, or anything covered by a customer contract that restricts subprocessors, a direct vendor connection removes the whole middle of the chain. That is often worth paying list price for.

If your workload is public documentation, code that is already open source, or content you would publish anyway, the calculus is completely different and a relay service is a reasonable choice.

Related reading

• Corporate procurement of a relay service: invoices, contracts, compliance and risk control (guides)
• Account throttled or banned: the common triggers for relay service risk controls (guides)
• How to check whether a relay service is quietly downgrading your model (guides)