RubyLLM (gateway) | Respan Docs

RubyLLM does not have a Ruby-side tracing instrumentor. Route all calls through the Respan gateway to capture every request as a trace.

Setup

Install RubyLLM

$ gem install ruby_llm

Or add it to your Gemfile.

1 gem "ruby_llm"

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No provider key needed. The Respan gateway handles provider authentication.

Configure RubyLLM with Respan

1 RubyLLM.configure do |config|
2   config.openai_api_key = ENV["RESPAN_API_KEY"]
3   config.openai_api_base = "https://api.respan.ai/api"
4 end

Make your first request

1 chat = RubyLLM.chat(model: "gpt-5.5")
2 response = chat.ask("Hello, world!")
3 puts response.content

View your trace

Open the Traces page to see your gateway-routed calls with prompts, tokens, and cost.

Switch models

Use another OpenAI model ID that Respan exposes through the same OpenAI-compatible endpoint.

1 chat = RubyLLM.chat(model: "gpt-5.5")
2 chat = RubyLLM.chat(model: "gpt-5-mini")
3 
4 response = chat.ask("Tell me about artificial intelligence")
5 puts response.content

RubyLLM’s OpenAI-compatible adapter does not provide a provider-neutral way to show Claude or Gemini model switches in this setup. Use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See the full model list.

Streaming

1 chat = RubyLLM.chat(model: "gpt-5.5")
2 chat.ask("Explain quantum computing") do |chunk|
3   print chunk.content
4 end

Responses API (reasoning, tools, and prompts)

RubyLLM’s default OpenAI provider uses the Chat Completions API. OpenAI blocks reasoning together with function tools on that endpoint (Function tools with reasoning_effort are not supported ... use /v1/responses instead). To combine reasoning and tool calls, route through the OpenAI Responses API with the ruby_llm-responses_api provider, pointed at the Respan gateway.

Install the provider gem

1 # Gemfile
2 gem "ruby_llm"
3 gem "ruby_llm-responses_api"

Point the Responses provider at Respan

1 require "ruby_llm"
2 require "ruby_llm-responses_api"
3 
4 RubyLLM.configure do |config|
5   config.openai_api_key  = ENV["RESPAN_API_KEY"]
6   config.openai_api_base = "https://api.respan.ai/api"
7 end
8 
9 # provider: :openai_responses routes to https://api.respan.ai/api/responses
10 chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
11 puts chat.ask("Hello!").content

OpenAI reasoning models (gpt-5.x, o-series) only accept temperature = 1 and reject top_p. Set with_temperature(1) on Responses chats that use these models, otherwise RubyLLM’s default temperature is rejected.

Reasoning with tool calls

Define a tool as usual, then combine with_thinking and with_tools. RubyLLM runs the tool loop for you.

1 class GetOrderStatus < RubyLLM::Tool
2   description "Look up an order's status, carrier, and tracking by order_id."
3   param :order_id, desc: "The order ID, e.g. 'A-48217'."
4 
5   def execute(order_id:)
6     { order_id:, status: "delayed", carrier: "UPS", tracking: "1Z999AA10123456784" }
7   end
8 end
9 
10 chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
11               .with_temperature(1)
12               .with_thinking(effort: :low)
13               .with_tools(GetOrderStatus)
14 
15 puts chat.ask("Where is my order A-48217?").content

Prompt management

Pass a Respan-managed prompt under respan_params with with_params. Do not use a top-level prompt key on the Responses path: the OpenAI Responses API has its own native prompt parameter, so a top-level prompt collides with it and the request fails.

1 chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
2               .with_params(
3                 respan_params: {
4                   prompt: {
5                     prompt_id: "YOUR_PROMPT_ID",
6                     variables: { store_name: "Lumen Electronics" }
7                   }
8                 }
9               )
10 
11 puts chat.ask("Hi, where is my order A-48217?").content

Prompt, tools, and reasoning together

The Responses path forwards the prompt’s stored parameters. When the prompt is used with a reasoning model, drop the stored top_p (which reasoning models reject) with a schema v2 patch.

1 chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
2               .with_temperature(1)          # reasoning models accept only temperature = 1
3               .with_thinking(effort: :low)  # reasoning
4               .with_tools(GetOrderStatus)   # tools (client-side)
5               .with_params(                 # Respan-managed prompt
6                 respan_params: {
7                   prompt: {
8                     prompt_id: "YOUR_PROMPT_ID",
9                     variables: { store_name: "Lumen Electronics" },
10                     schema_version: 2,
11                     patch: { top_p: nil }   # reasoning models reject top_p
12                   }
13                 }
14               )
15 
16 puts chat.ask("Hi, where is my order A-48217? It still hasn't arrived.").content

respan_params also accepts every other gateway field (customer_identifier, metadata, fallback_models, and so on). Prompt evaluation reruns stay on the Chat Completions path, but production traffic sent through the Responses API is still logged and can be evaluated as production data.

Multi-tenancy with contexts

Use RubyLLM contexts to isolate per-tenant configuration.

1 tenant_ctx = RubyLLM.context do |config|
2   config.openai_api_key = tenant.respan_api_key
3   config.openai_api_base = "https://api.respan.ai/api"
4 end
5 
6 chat = tenant_ctx.chat(model: "gpt-5.5")
7 response = chat.ask("Hello!")

Rails integration

Set your Respan config in an initializer.

1 # config/initializers/ruby_llm.rb
2 RubyLLM.configure do |config|
3   config.openai_api_key = ENV["RESPAN_API_KEY"]
4   config.openai_api_base = "https://api.respan.ai/api"
5 end

Use acts_as_chat as normal, and all LLM calls will be routed through Respan.