Together AI (tracing)

The Together AI SDKs are official clients for Together’s inference platform. respan-instrumentation-together and @respan/instrumentation-together-ai emit Respan spans for direct Together AI calls.

Set up Respan

Sign up - Create an account at platform.respan.ai
Create an API key - Generate one on the API keys page

Paste this into your coding agent:


Help me set up Respan tracing with the Together AI integration.
Follow https://www.respan.ai/docs/integrations/together-ai.md

Setup

Install packages

Terminal


pip install respan-ai respan-instrumentation-together together python-dotenv

Set environment variables

Terminal


export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export TOGETHER_API_KEY="YOUR_TOGETHER_API_KEY"

Optional:

Terminal


export TOGETHER_MODEL="meta-llama/Llama-3.2-3B-Instruct-Turbo"

TOGETHER_API_KEY is used by the Together AI SDK for direct provider calls. RESPAN_API_KEY is used by Respan for trace export.

Initialize and run

Python


import os

from dotenv import load_dotenv
from respan import Respan, workflow
from respan_instrumentation_together import TogetherInstrumentor
from together import Together

load_dotenv()

respan = Respan(
    api_key=os.environ["RESPAN_API_KEY"],
    instrumentations=[TogetherInstrumentor()],
)
client = Together(api_key=os.environ["TOGETHER_API_KEY"])


@workflow(name="together_chat_completion")
def run_chat() -> str:
    response = client.chat.completions.create(
        model=os.getenv("TOGETHER_MODEL", "meta-llama/Llama-3.2-3B-Instruct-Turbo"),
        messages=[
            {
                "role": "user",
                "content": "Reply with one concise sentence about tracing.",
            }
        ],
    )
    return response.choices[0].message.content or ""


try:
    print(run_chat())
finally:
    respan.shutdown()

View your trace

Open the Traces page and search for the workflow name together_chat_completion.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Respan API key. Falls back to `RESPAN_API_KEY`.
`base_url`	`str \| None`	`None`	Respan API base URL. Falls back to `RESPAN_BASE_URL`.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate, such as `TogetherInstrumentor()` or `new TogetherAIInstrumentor()`.
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all exported spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all exported spans.
`environment`	`str \| None`	`None`	Environment tag, such as `"production"`.

Supported calls

SDK call	Traced
`client.chat.completions.create(...)`	Chat completion spans
`client.chat.completions.create(..., stream=True)` / `stream: true`	Streaming chat completion spans after the stream is consumed or closed
`await async_client.chat.completions.create(...)`	Async chat completion spans
`client.completions.create(...)`	Text completion spans
`client.embeddings.create(...)`	Embedding spans with vector payloads summarized
`client.rerank.create(...)`	Rerank spans
`client.images.generate(...)`	Image generation spans
`tools=[...]` / `tool_calls` responses	Tool definitions and assistant tool calls

Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Together AI calls with propagate_attributes or propagateAttributes.

Python


from respan import propagate_attributes

with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_456",
    trace_group_identifier="together_support_chat",
    metadata={"plan": "pro", "workflow_name": "together_support_chat"},
):
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
        messages=[{"role": "user", "content": "Summarize our support policy."}],
    )

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`trace_group_identifier`	`str`	Groups spans by workflow name.
`metadata`	`dict`	Custom key-value pairs merged with default metadata.

Examples

Streaming chat

Python


stream = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Write a short haiku about traces."}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Embeddings

Python


response = client.embeddings.create(
    model="BAAI/bge-base-en-v1.5",
    input=[
        "Respan traces Together AI chat calls.",
        "Embeddings should not export vector payloads.",
    ],
)
print(len(response.data))

Tool calling

Python


response = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
    messages=[{"role": "user", "content": "What is the weather in Tokyo?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city.",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
)
print(response.choices[0].message.tool_calls)