> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.respan.ai/docs/_mcp/server.

# Together AI (gateway)

> Set up Together AI model calls with the Respan gateway.

Route Together AI model calls through the Respan gateway to use Respan credentials, request logs, routing, fallbacks, and metadata. No `TOGETHER_API_KEY` is needed in application code when TogetherAI is configured in Respan.

## Setup

#### Install packages

**`Python`**

```bash Python
pip install openai python-dotenv
```

**`TypeScript`**

```bash TypeScript
npm install openai dotenv
```

#### Set environment variables

```bash
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
```

Optional:

```bash
export RESPAN_BASE_URL="https://api.respan.ai/api"
```

Together AI requires **bring your own key (BYOK)**. Add your Together AI key on the [Providers page](https://platform.respan.ai/platform/gateway/api-keys?providers=1) before making requests. Together AI usage cannot be paid with Respan credits. Once the key is stored in Respan, you can keep `TOGETHER_API_KEY` out of your application runtime.

#### Point an OpenAI-compatible client to the Respan gateway

**`Python`**

```python Python
import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
)

response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)
```

**`TypeScript`**

```typescript TypeScript
import "dotenv/config";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.RESPAN_API_KEY,
  baseURL: process.env.RESPAN_BASE_URL ?? "https://api.respan.ai/api",
});

const response = await client.chat.completions.create({
  model: "together_ai/Qwen/Qwen3.8-Flash",
  messages: [{ role: "user", content: "Say hello in three languages." }],
});
console.log(response.choices[0]?.message?.content);
```

## Switch models

Change the `model` parameter to use 1000+ models from different providers through the same gateway.

**`Python`**

```python Python
response = client.chat.completions.create(model="together_ai/Qwen/Qwen3.8-Flash", messages=messages)
response = client.chat.completions.create(model="together_ai/deepseek-ai/DeepSeek-V4.1-Flash", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
```

**`TypeScript`**

```typescript TypeScript
await client.chat.completions.create({ model: "together_ai/Qwen/Qwen3.8-Flash", messages });
await client.chat.completions.create({ model: "together_ai/deepseek-ai/DeepSeek-V4.1-Flash", messages });
await client.chat.completions.create({ model: "openai/gpt-6-sol", messages });
```

See the [full model list](https://platform.respan.ai/platform/models).

## Respan parameters

Pass additional Respan parameters via `extra_body` for gateway features.

**`Python`**

```python Python
response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["openai/gpt-6-sol"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)
```

**`TypeScript`**

```typescript TypeScript
const response = await client.chat.completions.create({
  model: "together_ai/Qwen/Qwen3.8-Flash",
  messages: [{ role: "user", content: "Hello" }],
  customer_identifier: "user_123",
  fallback_models: ["openai/gpt-6-sol"],
  metadata: { session_id: "abc123" },
  thread_identifier: "conversation_456",
});
```

See [Respan params & metadata](/docs/documentation/features/gateway/respan-params) for the full list.

## Bring your own Together AI key

Your Together AI key is required, and Together AI bills provider usage to your account. Store the key in Respan or pass it per request as shown below. The provider ID for request-level credentials is `togetherai` and the credential field is `api_key`.

**`Python`**

```python Python
response = client.chat.completions.create(
    model="together_ai/Qwen/Qwen3.8-Flash",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_credentials": {
            "togetherai": {
                "api_key": "YOUR_TOGETHER_API_KEY"
            }
        }
    },
)
```

**`TypeScript`**

```typescript TypeScript
const response = await client.chat.completions.create({
  model: "together_ai/Qwen/Qwen3.8-Flash",
  messages: [{ role: "user", content: "Hello" }],
  customer_credentials: {
    togetherai: {
      api_key: "YOUR_TOGETHER_API_KEY",
    },
  },
});
```