GatewayAgent Frameworks

Guardrails (gateway)

Route Guardrails’ underlying LLM calls through the Respan gateway to use 1000+ models from different providers. Only your RESPAN_API_KEY is needed — no separate provider keys required.

Setup

1

Install packages

pip install guardrails-ai openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export OPENAI_API_KEY="YOUR_RESPAN_API_KEY"
export OPENAI_BASE_URL="https://api.respan.ai/api"

No real OpenAI key needed — the Respan gateway handles provider authentication.

3

Run your guard

import os
from typing import Literal
os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
os.environ["OPENAI_BASE_URL"] = "https://api.respan.ai/api"
from guardrails import Guard
from pydantic import BaseModel, Field
class SupportReply(BaseModel):
    answer: str = Field(description="Customer-facing answer")
    priority: Literal["low", "medium", "high"] = Field(description="Ticket priority")
guard = Guard.for_pydantic(output_class=SupportReply)
result = guard(
    model="gpt-5-mini",
    messages=[
        {
            "role": "user",
            "content": "Return JSON with answer and priority for a delayed shipment.",
        }
    ],
)
print(result.validated_output)

Switch models

Change the model parameter on guard(...) to use another OpenAI model through the same gateway-backed endpoint.

result = guard(model="gpt-5-mini", messages=messages)
result = guard(model="gpt-5.5", messages=messages)

Guardrails routes through LiteLLM. Bare Claude or Gemini model strings make LiteLLM call those providers directly instead of the Respan OpenAI-compatible gateway, so this page keeps the switch examples to OpenAI models. Use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See the full model list.