Wiring a relay service into Dify or FastGPT

How it differs from writing the code yourself

Platforms like Dify and FastGPT already wrap model integration; you fill in a few fields in the UI. But attaching a relay service adds two complications:

1. The requests are more complex than you expect (lots of metadata, plus tool definitions)
2. Retry and concurrency policy belongs to the platform, not to you

Which means: a service where "curl works fine by hand" can be unstable once it sits behind the platform.

Configuration steps (general)

1. Choose "OpenAI compatible" or "custom provider"

Most platforms offer an "OpenAI-compatible endpoint" option — the least painful way to attach a relay service.

Fill in three fields:

API Base:  https://service-domain/v1
API Key:   your key
Model:     the id the service offers

2. Test connectivity before building an app

Platforms usually have a "Test connection" button. Make the model list come back first, then build your chat app.

If the model list will not load: check whether base_url needs the /v1.

3. Watch the model's capability declaration

Some platforms make you declare model capabilities by hand (function calling, vision). Declaring them wrong leads to:

• You ticked "supports function calling" but the service does not → tool calls fail inside the workflow
• You left it unticked but the model does support it → the platform never uses the capability

Go by what you actually test, not by a copy of the official capability table — relay services implement these features to varying degrees.

Three limits that get overlooked

1. Platform retries inflate the bill

When a node in the workflow fails, the platform usually retries. If a service is unstable, retries mean:

• A higher bill (a retry is still a request)
• Longer response times (waiting out the retry)

Suggested: set a retry ceiling in the workflow, or move unstable nodes to a steadier service.

2. The platform may not support custom request parameters

You want to use a parameter specific to one service (a thinking budget, say), but the platform gives you nowhere to pass it.

Fix: look for a "custom model parameters" field (some platforms have one), or accept the defaults.

3. Knowledge-base retrieval multiplies token use

A RAG platform stuffs retrieved documents into the context on every turn. That content repeats heavily, and depends on caching.

If the service does not cache, a knowledge-base app's bill will be far higher than expected.

How to check: use a test account that asks the same thing repeatedly, and watch how fast the balance drains.

A practical way to evaluate

Do not judge a service by the platform's built-in ping. Evaluate it with a real workflow:

1. Build a complete workflow with knowledge-base retrieval plus a tool call
2. Run 20 real questions through it
3. Record: success rate, average response time, balance consumed

Those three numbers are worth more than any marketing claim.

On "which service to use"

These platforms are usually team environments, so:

• Pick a service with good stability and concurrency, not the cheapest
• Configure two providers (most platforms allow it) so you can switch when one misbehaves
• Watch the platform's call statistics and reconcile them against the service's bill regularly

Related reading

• Team scenarios: managing one shared key across several people
• Does prompt caching actually work on a relay service?
• How to give a project several API providers as failover