Switching base_url in the Python SDK: five things people miss

Is it really a one-line change? Mostly

from openai import OpenAI

client = OpenAI(
    base_url="https://service-domain/v1",
    api_key="your-key",
)

But these five things will find you later.

1. Should base_url end in /v1?

This is the most common question.

What the SDK does: it joins base_url with the specific path. So:

# Correct (most of the time)
base_url="https://service-domain/v1"
# Actual request -> https://service-domain/v1/chat/completions

# If the service wants no v1
base_url="https://service-domain"
# Actual request -> https://service-domain/chat/completions

How to tell: try both and see which works. Or just follow the example in the service's own docs.

2. Timeout settings

The SDK default timeout is usually short (tens of seconds to a few minutes). But:

• A reasoning model's response can take more than 5 minutes
• Long-document processing can be slow too

Suggested: set timeouts per request type rather than one global value.

client = OpenAI(
    base_url=...,
    api_key=...,
    timeout=60.0,      # ordinary requests
    max_retries=2,     # set retries explicitly
)

Note max_retries: the SDK retries automatically by default. If a service is unstable, blind retries inflate the bill. In production, set it explicitly.

3. Retry policy and idempotency

Automatic retry sounds safe, but consider:

• Will a retried request be billed twice? (Most services will not, but a partially successful call may already have been billed)
• Is your operation idempotent? (Retrying a generation produces several results)

Suggested: set an explicit retry count (2, say) and log retries so you can reconcile afterwards.

4. Environment variables versus hard-coding

# Not recommended -- switching services means editing code and redeploying
client = OpenAI(base_url="https://some-service.com/v1", api_key="sk-xxx")

# Recommended -- switching services is a config change
import os
client = OpenAI(
    base_url=os.environ["API_BASE_URL"],
    api_key=os.environ["API_KEY"],
)

This is the most important item here. Making base_url a configuration value means:

• Switching services does not require a code change
• You can do multi-service routing, canaries, failover
• If a service disappears, the damage is limited to your balance, not your product

5. Concurrency control

The SDK does not limit concurrency at all — you get as much as your code asks for. That gets out of hand easily in async code.

# Limit concurrency with a semaphore
import asyncio
sem = asyncio.Semaphore(5)   # at most 5 concurrent

async def call_api(...):
    async with sem:
        return await client.chat.completions.create(...)

Why it matters: most relay services cap concurrency per account. Exceed it and you do not get a "concurrency exceeded" error — you get random partial failures, which are painful to diagnose.

A complete configuration template

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("API_BASE_URL", "https://default-service/v1"),
    api_key=os.environ["API_KEY"],
    timeout=float(os.environ.get("API_TIMEOUT", "60")),
    max_retries=int(os.environ.get("API_MAX_RETRIES", "2")),
)

# Suggested: log the base_url actually in use, so you can confirm the channel when reconciling
import logging
logging.info("API base_url = %s", client.base_url)

That last log line earns its keep: with several services in play, when something breaks you know immediately which channel you were on.

Related reading

• A relay service suddenly errors or slows down: how to tell whose problem it is
• How to give a project several API providers as failover
• Rate-limited or banned: what actually triggers a relay service's risk controls