LlamaIndex (gateway)

Route LlamaIndex LLM and embedding calls through the Respan gateway to use 1000+ models from different providers. Only your Respan API key is needed. No separate provider key is required when the provider is configured in Respan.

Paste this into your coding agent:


Help me set up the Respan gateway with the LlamaIndex integration.
Follow https://www.respan.ai/docs/integrations/gateway/llama-index.md

Setup

Install packages

Terminal


pip install llama-index llama-index-llms-openai llama-index-embeddings-openai

Set environment variables

Terminal


export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Point LlamaIndex to the Respan gateway

Python


import os
from llama_index.core import Document, Settings, SummaryIndex
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI

respan_api_key = os.environ["RESPAN_API_KEY"]
respan_base_url = "https://api.respan.ai/api"

Settings.llm = OpenAI(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="gpt-5-mini",
)
Settings.embed_model = OpenAIEmbedding(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="text-embedding-3-small",
)

index = SummaryIndex.from_documents([
    Document(text="The Respan gateway routes LlamaIndex calls to hosted models.")
])
response = index.as_query_engine().query("What does the gateway do?")
print(response)

Switch models

Change the model parameter to use another OpenAI model through the same gateway-backed endpoint.

Python


Settings.llm = OpenAI(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="gpt-5.5",
)

OpenAI is LlamaIndex’s OpenAI-compatible LLM adapter. This page avoids showing Claude or Gemini inside that OpenAI-named adapter; use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See Respan params & metadata for the full list.