Skip to main content
  1. Sign up — Create an account at platform.respan.ai
  2. Create an API key — Generate one on the API keys page
  3. Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page
This integration is for the Respan gateway.

​
Overview

LlamaIndex provides a powerful framework for building LLM applications with data. You can seamlessly integrate Respan with LlamaIndex’s OpenAI LLM with minimal code changes.

​
Quickstart

​
Step 1: Install LlamaIndex

pip install llama-index-llms-openai

​
Step 2: Initialize LlamaIndex with Respan

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    api_base="https://api.respan.ai/api/",
    api_key="<Your Respan API Key>",
    model="gpt-3.5-turbo"
)

​
Step 3: Make Your First Request

response = llm.complete("Hello, world!")
print(response)

​
Switch models

# OpenAI GPT models
model = "gpt-4o"
# model = "claude-3-5-sonnet-20241022"
# model = "gemini-1.5-pro"

llm = OpenAI(
    api_base="https://api.respan.ai/api/",
    api_key="<Your Respan API Key>",
    model=model
)
See the full model list for all available models.

​
Supported parameters

​
OpenAI parameters

We support all the OpenAI parameters. You can pass them directly in the LlamaIndex configuration. 
llm = OpenAI(
    api_base="https://api.respan.ai/api/",
    api_key="<Your Respan API Key>",
    model="gpt-4o-mini",
    temperature=0.7,          # Control randomness
    max_tokens=1000,          # Limit response length
)

​
Respan Parameters

Respan parameters can be passed using extra_body for better handling and customization. 
response = llm.complete(
    "Tell me a story",
    extra_body={
        "customer_identifier": "user_123",           # Track specific users
        "fallback_models": ["gpt-3.5-turbo"],       # Automatic fallbacks
        "metadata": {"session_id": "abc123"},        # Custom metadata
        "thread_identifier": "conversation_456",     # Group related messages
        "group_identifier": "team_alpha",           # Organize by groups
    }
)

View your analytics

Access your Respan dashboard to see detailed analytics

​
Next Steps

Advanced settings

Track user behavior and patterns

Prompt Management

Manage and version your prompts