GatewayAgent Frameworks

Haystack (gateway)

Route Haystack’s OpenAI-compatible LLM calls through the Respan gateway to use 250+ models from different providers. No separate OpenAI provider key is required.

Setup

1

Install packages

$pip install haystack-ai
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Point Haystack to the Respan gateway

1import os
2
3from haystack import Pipeline
4from haystack.components.builders import PromptBuilder
5from haystack.components.generators import OpenAIGenerator
6
7os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
8os.environ["OPENAI_BASE_URL"] = os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api")
9
10pipeline = Pipeline()
11pipeline.add_component(
12    "prompt_builder",
13    PromptBuilder(template="Answer the following question: {{question}}"),
14)
15pipeline.add_component("generator", OpenAIGenerator(model="gpt-5-mini"))
16pipeline.connect("prompt_builder", "generator")
17
18result = pipeline.run(
19    {"prompt_builder": {"question": "What is the capital of France?"}}
20)
21print(result["generator"]["replies"][0])

Switch models

Change the model parameter on OpenAIGenerator to use another OpenAI model through the same gateway-backed endpoint.

1pipeline.add_component(
2    "generator",
3    OpenAIGenerator(model="gpt-5.5"),
4)

OpenAIGenerator is Haystack’s OpenAI-compatible generator. This page avoids showing Claude or Gemini inside that OpenAI-named component; use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See the full model list.