Provider: Groq

Call Groq models through Respan Gateway with unified logs, cost, and latency.

This page is for Respan LLM Gateway users.

Use Respan Gateway to call Groq models (groq/qwen/qwen3.8-27b, groq/qwen/qwen3.6-27b, groq/openai/gpt-oss-120b, and the rest) while keeping unified observability (logs, cost, latency, reliability) in Respan.

Groq requires bring your own key (BYOK). Provider usage cannot be paid with Respan credits.

Paste this into your coding agent:


Help me set up the Respan gateway to call Groq models.
Follow https://www.respan.ai/docs/integrations/gateway/model-providers/groq.md

Quick setup

Get a Respan API key

Add your Groq credentials

Configure your Groq credentials on the Providers page before using the examples below. See Use your own Groq key.

Send your first request

After configuring your Groq credentials in Respan, pick the integration that matches your stack. The base URL is https://api.respan.ai/api and your application authenticates with RESPAN_API_KEY.

Point the official Groq SDK at the Respan gateway by overriding base_url.


from groq import Groq

client = Groq(
    api_key="YOUR_RESPAN_API_KEY",
    base_url="https://api.respan.ai/api",
)

response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello, Groq!"}],
)
print(response.choices[0].message.content)

Groq models are OpenAI-compatible. Point the OpenAI SDK at Respan and use a Groq model ID.


from openai import OpenAI

client = OpenAI(
    api_key="YOUR_RESPAN_API_KEY",
    base_url="https://api.respan.ai/api",
)

response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello, Groq!"}],
)
print(response.choices[0].message.content)

Call the Respan REST API directly from any language.


import requests

response = requests.post(
    "https://api.respan.ai/api/chat/completions",
    headers={
        "Authorization": "Bearer YOUR_RESPAN_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "groq/qwen/qwen3.8-27b",
        "messages": [{"role": "user", "content": "Hello, Groq!"}],
    },
)
print(response.json()["choices"][0]["message"]["content"])

More integrations

Groq models work with every Respan gateway integration:

Switch models

Change the model parameter to call any supported model through the same client. Use the groq/ prefix to disambiguate when routing across providers. Browse the full list on the Models page .


client.chat.completions.create(model="groq/qwen/qwen3.8-27b", messages=messages)
client.chat.completions.create(model="groq/qwen/qwen3.6-27b", messages=messages)
client.chat.completions.create(model="groq/openai/gpt-oss-120b", messages=messages)
client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
client.chat.completions.create(model="anthropic/claude-opus-5-5", messages=messages)

Use your own Groq key (BYOK)

Groq credentials are required. Configure them in Respan or pass them with each request using one of the options below.

Override credentials per model (Optional)

Use credential_override when one model on a request should use a different Groq key than the default.


{
  "customer_credentials": {
    "groq": { "api_key": "YOUR_GROQ_API_KEY" }
  },
  "credential_override": {
    "groq/qwen/qwen3.6-27b": { "api_key": "ANOTHER_GROQ_API_KEY" }
  }
}

Log without proxying (Optional)

Already calling Groq directly? Send logs to Respan asynchronously to track cost, latency, and performance for those external calls.


import requests

requests.post(
    "https://api.respan.ai/api/request-logs/create/",
    headers={
        "Authorization": "Bearer YOUR_RESPAN_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "groq/qwen/qwen3.8-27b",
        "prompt_messages": [{"role": "user", "content": "Hello, how are you?"}],
        "completion_message": {"role": "assistant", "content": "Hello from Groq through Respan."},
        "cost": 0.001,
        "generation_time": 1.2,
        "customer_params": {"customer_identifier": "user_123"},
    },
)

See the logging guide for the full setup.