LlamaIndex | Respan Docs

Set up Respan

Sign up — Create an account at platform.respan.ai
Create an API key — Generate one on the API keys page
Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Use AI

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1 {
2   "mcpServers": {
3     "respan-docs": {
4       "url": "https://mcp.respan.ai/mcp/docs"
5     }
6   }
7 }

This integration is for the Respan gateway.

What is LlamaIndex?

LlamaIndex provides a powerful framework for building LLM applications with data. You can seamlessly integrate Respan with LlamaIndex’s OpenAI LLM with minimal code changes.

Quickstart

Step 1: Install LlamaIndex

Python

$ pip install llama-index-llms-openai

Step 2: Initialize LlamaIndex with Respan

Python

1 from llama_index.llms.openai import OpenAI
2 
3 llm = OpenAI(
4     api_base="https://api.respan.ai/api/",
5     api_key="<Your Respan API Key>",
6     model="gpt-3.5-turbo"
7 )

Step 3: Make Your First Request

Python

1 response = llm.complete("Hello, world!")
2 print(response)

Switch models

Python

1 # OpenAI GPT models
2 model = "gpt-4o"
3 # model = "claude-3-5-sonnet-20241022"
4 # model = "gemini-1.5-pro"
5 
6 llm = OpenAI(
7     api_base="https://api.respan.ai/api/",
8     api_key="<Your Respan API Key>",
9     model=model
10 )

See the full model list for all available models.

Supported parameters

OpenAI parameters

We support all the OpenAI parameters. You can pass them directly in the LlamaIndex configuration.

Python

1 llm = OpenAI(
2     api_base="https://api.respan.ai/api/",
3     api_key="<Your Respan API Key>",
4     model="gpt-4o-mini",
5     temperature=0.7,          # Control randomness
6     max_tokens=1000,          # Limit response length
7 )

Respan Parameters

Respan parameters can be passed using extra_body for better handling and customization.

Python

1 response = llm.complete(
2     "Tell me a story",
3     extra_body={
4         "customer_identifier": "user_123",           # Track specific users
5         "fallback_models": ["gpt-3.5-turbo"],       # Automatic fallbacks
6         "metadata": {"session_id": "abc123"},        # Custom metadata
7         "thread_identifier": "conversation_456",     # Group related messages
8         "group_identifier": "team_alpha",           # Organize by groups
9     }
10 )

View your analytics

Access your Respan dashboard to see detailed analytics

Next Steps

Advanced settings

Track user behavior and patterns

Prompt Management

Manage and version your prompts