Groq (tracing) | Respan Docs

The Groq Python SDK is the official client for Groq’s inference API. respan-instrumentation-groq activates the Groq OpenInference wrapper and normalizes emitted spans to the Respan tracing pipeline.

Set up Respan

Sign up - Create an account at platform.respan.ai
Create an API key - Generate one on the API keys page

Use Respan Gateway

See Groq gateway setup to route Groq model calls through the Respan gateway.

Example projects

Respan example projects

Setup

Install packages

$ pip install respan-ai respan-instrumentation-groq groq python-dotenv

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
$ export GROQ_API_KEY="YOUR_GROQ_API_KEY"

Optional:

$ export RESPAN_BASE_URL="https://api.respan.ai/api"
$ export GROQ_MODEL="llama-3.1-8b-instant"

Initialize and run

1 import os
2 
3 from dotenv import load_dotenv
4 from groq import Groq
5 from respan import Respan, workflow
6 from respan_instrumentation_groq import GroqInstrumentor
7 
8 load_dotenv()
9 
10 respan = Respan(
11     api_key=os.environ["RESPAN_API_KEY"],
12     base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
13     instrumentations=[GroqInstrumentor()],
14 )
15 client = Groq(api_key=os.environ["GROQ_API_KEY"])
16 
17 
18 @workflow(name="groq_chat_completion")
19 def run_chat() -> str:
20     response = client.chat.completions.create(
21         model=os.getenv("GROQ_MODEL", "llama-3.1-8b-instant"),
22         messages=[
23             {
24                 "role": "user",
25                 "content": "Reply with one concise sentence about tracing.",
26             }
27         ],
28     )
29     return response.choices[0].message.content or ""
30 
31 
32 try:
33     print(run_chat())
34 finally:
35     respan.shutdown()

View your trace

Open the Traces page and search for the workflow name groq_chat_completion.

Configuration

Parameter	Type	Default	Description
`**instrumentor_kwargs`	`dict`	`{}`	Extra keyword arguments forwarded to the upstream Groq OpenInference instrumentor.

Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Groq calls with propagate_attributes.

1 from respan import propagate_attributes
2 
3 with propagate_attributes(
4     customer_identifier="user_123",
5     thread_identifier="conversation_456",
6     trace_group_identifier="groq_support_chat.workflow",
7     metadata={"plan": "pro", "workflow_name": "groq_support_chat.workflow"},
8 ):
9     response = client.chat.completions.create(
10         model="llama-3.1-8b-instant",
11         messages=[{"role": "user", "content": "Summarize our support policy."}],
12     )

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`trace_group_identifier`	`str`	Groups spans by workflow name.
`metadata`	`dict`	Custom key-value pairs merged with default metadata.

Examples

Streaming

1 stream = client.chat.completions.create(
2     model="llama-3.1-8b-instant",
3     messages=[{"role": "user", "content": "Write a short haiku about tracing."}],
4     stream=True,
5 )
6 
7 for chunk in stream:
8     content = chunk.choices[0].delta.content
9     if content:
10         print(content, end="", flush=True)

Tool calling

1 response = client.chat.completions.create(
2     model="llama-3.1-8b-instant",
3     messages=[{"role": "user", "content": "What is the weather in Boston?"}],
4     tools=[
5         {
6             "type": "function",
7             "function": {
8                 "name": "get_weather",
9                 "description": "Get the current weather for a city.",
10                 "parameters": {
11                     "type": "object",
12                     "properties": {"city": {"type": "string"}},
13                     "required": ["city"],
14                 },
15             },
16         }
17     ],
18 )
19 print(response.choices[0].message.tool_calls)