GatewayAgent Frameworks

LlamaIndex (gateway)

Route LlamaIndex LLM and embedding calls through the Respan gateway to use 1000+ models from different providers. Only your Respan API key is needed. No separate provider key is required when the provider is configured in Respan.

Setup

1

Install packages

pip install llama-index llama-index-llms-openai llama-index-embeddings-openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Point LlamaIndex to the Respan gateway

import os
from llama_index.core import Document, Settings, SummaryIndex
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
respan_api_key = os.environ["RESPAN_API_KEY"]
respan_base_url = "https://api.respan.ai/api"
Settings.llm = OpenAI(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="gpt-5-mini",
)
Settings.embed_model = OpenAIEmbedding(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="text-embedding-3-small",
)
index = SummaryIndex.from_documents([
    Document(text="The Respan gateway routes LlamaIndex calls to hosted models.")
])
response = index.as_query_engine().query("What does the gateway do?")
print(response)

Switch models

Change the model parameter to use another OpenAI model through the same gateway-backed endpoint.

Settings.llm = OpenAI(
    api_key=respan_api_key,
    api_base=respan_base_url,
    model="gpt-5.5",
)

OpenAI is LlamaIndex’s OpenAI-compatible LLM adapter. This page avoids showing Claude or Gemini inside that OpenAI-named adapter; use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See Respan params & metadata for the full list.