Guardrails (gateway)

Route Guardrails’ underlying LLM calls through the Respan gateway to use 250+ models from different providers. Only your RESPAN_API_KEY is needed — no separate provider keys required.

Setup

Install packages

$ pip install guardrails-ai openai

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
$ export OPENAI_API_KEY="YOUR_RESPAN_API_KEY"
$ export OPENAI_BASE_URL="https://api.respan.ai/api"

No real OpenAI key needed — the Respan gateway handles provider authentication.

Run your guard

1 import os
2 from typing import Literal
3 
4 os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
5 os.environ["OPENAI_BASE_URL"] = "https://api.respan.ai/api"
6 
7 from guardrails import Guard
8 from pydantic import BaseModel, Field
9 
10 
11 class SupportReply(BaseModel):
12     answer: str = Field(description="Customer-facing answer")
13     priority: Literal["low", "medium", "high"] = Field(description="Ticket priority")
14 
15 
16 guard = Guard.for_pydantic(output_class=SupportReply)
17 
18 result = guard(
19     model="gpt-5-mini",
20     messages=[
21         {
22             "role": "user",
23             "content": "Return JSON with answer and priority for a delayed shipment.",
24         }
25     ],
26 )
27 print(result.validated_output)

Switch models

Change the model parameter on guard(...) to use another OpenAI model through the same gateway-backed endpoint.

1 result = guard(model="gpt-5-mini", messages=messages)
2 result = guard(model="gpt-5.5", messages=messages)

Guardrails routes through LiteLLM. Bare Claude or Gemini model strings make LiteLLM call those providers directly instead of the Respan OpenAI-compatible gateway, so this page keeps the switch examples to OpenAI models. Use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See the full model list.