Together AI (gateway)

Route Together AI model calls through the Respan gateway to use Respan credentials, request logs, routing, fallbacks, and metadata. No TOGETHER_API_KEY is needed in application code when TogetherAI is configured in Respan.

Setup

Install packages

pip install openai python-dotenv

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Optional:

export RESPAN_BASE_URL="https://api.respan.ai/api"

Together AI requires bring your own key (BYOK). Add your Together AI key on the Providers page before making requests. Together AI usage cannot be paid with Respan credits. Once the key is stored in Respan, you can keep TOGETHER_API_KEY out of your application runtime.

Point an OpenAI-compatible client to the Respan gateway

import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
)
response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="together_ai/Qwen/Qwen3.8-Flash", messages=messages)
response = client.chat.completions.create(model="together_ai/deepseek-ai/DeepSeek-V4.1-Flash", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)

See the full model list.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["openai/gpt-6-sol"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)

See Respan params & metadata for the full list.

Bring your own Together AI key

Your Together AI key is required, and Together AI bills provider usage to your account. Store the key in Respan or pass it per request as shown below. The provider ID for request-level credentials is togetherai and the credential field is api_key.

response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_credentials": {
            "togetherai": {
                "api_key": "YOUR_TOGETHER_API_KEY"
            }
        }
    },
)