Skip to navigation
IntegrationIntegrationsLLM SDKs

Groq (gateway)

Route Groq model calls through the Respan gateway to use Respan credentials, request logs, routing, fallbacks, and metadata. For direct Groq SDK tracing, see Groq tracing setup.

Setup

1

Install packages

pip install openai python-dotenv
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No GROQ_API_KEY is required for gateway calls when your Groq provider credentials are configured in Respan.

3

Point an OpenAI-compatible client to the Respan gateway

import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
)
response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="groq/qwen/qwen3.8-27b", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
response = client.chat.completions.create(model="anthropic/claude-opus-5-5", messages=messages)

See the full model list.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["openai/gpt-6-sol"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)

See Respan params & metadata for the full list.