Advanced

1
{
2
  "mcpServers": {
3
    "respan-docs": {
4
      "url": "https://docs.respan.ai/mcp"
5
    }
6
  }
7
}

Build, test, and deploy prompt templates on Respan. New to prompts? Start with the quickstart guide.

Prompt schema

The prompt object supports a schema_version field that controls how prompt configuration and request-body parameters are merged.

Prompt schema v2 (recommended)

Set schema_version=2 for the recommended merge behavior:

• Prompt configuration always wins for conflicting fields (no override flag needed).
• Uses prepend/instructions-style merging depending on the endpoint mode.
• Supports a patch field for applying additional parameter overrides. The patch object must not contain messages or input.

1
import requests
2
3
headers = {
4
    'Content-Type': 'application/json',
5
    'Authorization': 'Bearer YOUR_RESPAN_API_KEY',
6
}
7
8
data = {
9
    'prompt': {
10
        'prompt_id': 'YOUR_PROMPT_ID',
11
        'schema_version': 2,
12
        'variables': {
13
            'task_description': 'Square a number',
14
        },
15
        'patch': {
16
            'temperature': 0.9,
17
            'max_tokens': 500
18
        }
19
    }
20
}
21
22
response = requests.post(
23
    'https://api.respan.ai/api/chat/completions',
24
    headers=headers,
25
    json=data
26
)
27
print(response.json())

Prompt schema v1 (default, legacy)

When schema_version is absent or 1, merging is controlled by the override flag:

• override=true: prompt configuration wins for all conflicting fields.
• override=false (default): request body wins for conflicting fields.

1
request_body = {
2
    "prompt": {
3
        "prompt_id": "042f5f",
4
        "override": True,
5
        "override_params": {
6
            "temperature": 0.8,
7
            "max_tokens": 150,
8
            "model": "gpt-4o"
9
        }
10
    }
11
}

1
request_body = {
2
    "prompt": {
3
        "prompt_id": "042f5f",
4
        "override_config": {"messages_override_mode": "append"},
5
        "override_params": {"messages": [{"role": "user", "content": "Additional context"}]},
6
    }
7
}

1
request_body = {
2
    "prompt": {
3
        "prompt_id": "042f5f",
4
        "override_config": {"messages_override_mode": "override"},
5
        "override_params": {"messages": [{"role": "user", "content": "Completely new conversation"}]},
6
    }
7
}

Variables

Use {{variable_name}} syntax to add dynamic content to your prompts. Variables let you reuse the same template across different inputs — pass values at runtime from code, or fill them from testset columns in experiments. See the quickstart for setup and basic usage.

Jinja templates

Respan supports Jinja templates for conditionals, filters, and JSON input access.

See the Filters in Jinja templates guide for details.

You can also set default variable values when creating a version via the API:

1
data = {
2
    "messages": [
3
        {"role": "system", "content": "You are a helpful {{role}}."},
4
        {"role": "user", "content": "Help me with {{task_description}}."}
5
    ],
6
    "model": "gpt-4o-mini",
7
    "variables": {
8
        "role": "assistant",
9
        "task_description": "sorting an array"
10
    },
11
}

Prompt composition

Prompt composition lets a variable in one prompt reference another prompt. At request time, the child is rendered first, converted to plain text, and inserted into the parent variable.

To use prompt composition, create two prompts: a child prompt and a parent prompt that has a {{variable}} where the child’s output will be injected.

Use in UI

Use in code

1

Configure the variable

In the parent prompt editor:

• Open the Variables panel on the right side
• Find the variable you want to embed a prompt into
• Change its type from Text to Prompt
• Select the child prompt
• Fill in all the child’s variables

2

Run your prompt

Click Run to test. The child prompt is rendered first, converted to plain text, and injected into the parent variable before the LLM sees it.

Limits:

• Circular references are rejected (HTTP 400).
• Maximum prompt-chain depth is 2 (parent → child is safest). Exceeding this returns HTTP 400.

Structured output

Define structured output using JSON schema to ensure AI responses follow a specific format, following the OpenAI Structured Outputs specification.

Setup

Use in code

Configure JSON schema in the prompt editor or the playground:

You can generate schemas using the AI generator or browse examples in the editor.

Streaming

Enable streaming in the prompt settings sidebar. After enabling, commit and deploy the prompt.

If you use a prompt with streaming enabled, you must also set stream=True in your SDK call:

1
response = client.chat.completions.create(
2
    model="gpt-4o-mini",
3
    messages=[{"role":"user", "content":"Tell me a long story"}],
4
    stream=True,
5
    extra_body={
6
      "prompt": {
7
          "prompt_id": "YOUR_PROMPT_ID",
8
          "variables": {"variable_name": "variable_value"},
9
        }
10
    }
11
)

Fallback models

Add fallback models to automatically retry on a different model if the primary one fails. Pass fallback_models alongside your prompt config:

1
response = client.chat.completions.create(
2
    model="gpt-4o",
3
    messages=[{"role": "user", "content": "Hello"}],
4
    extra_body={
5
        "prompt": {
6
            "prompt_id": "my-prompt",
7
            "schema_version": 2,
8
        },
9
        "fallback_models": ["claude-sonnet-4-20250514", "gemini-2.5-flash"],
10
    },
11
)

The gateway tries each model in order until one succeeds. Combined with prompt management, this means your prompt template, model selection, and fallback strategy are all managed in one place.

Deployment & versioning

Via UI

Via code

Commit saves a new version of your prompt. Deploy makes a version live for production traffic.

View version history in the Overview panel, or click Version in the Editor for detailed diffs.

Click on each version to see the diff of changes.

To deploy, go to the Deployments tab and click Deploy.

To rollback, deploy an earlier version from the Deployments tab.

Playground

Test and iterate on prompts in the Prompt Playground.

From the prompt editor, enter variable values and click Playground in the top bar to test. Click Commit to save changes back to your prompt library.

You can also debug production logs by clicking Open in Playground on any log entry.

Set the number of variants in the side panel to generate multiple responses and compare them in the Variants tab.

Prompt logging

Respan prompts

External prompts

Log prompt usage to track performance metrics, compare versions, and analyze request distribution.

PythonTypeScript

1
import requests
2
3
url = "https://api.respan.ai/api/request-logs/create/"
4
payload = {
5
    "model": "claude-3-5-sonnet-20240620",
6
    "completion_message": {
7
        "role": "assistant",
8
        "content": "Hi, how can I assist you today?"
9
    },
10
    "prompt": {
11
        "prompt_id": "xxxxxx",
12
        "variables": {
13
            "task_description": "Square a number",
14
            "specific_library": "math"
15
        },
16
    },
17
    "generation_time": 5.7,
18
    "ttft": 3.1,
19
}
20
headers = {
21
    "Authorization": "Bearer YOUR_RESPAN_API_KEY",
22
    "Content-Type": "application/json"
23
}
24
25
response = requests.post(url, headers=headers, json=payload)

Filter logs by prompt name on the Logs page.

Team collaboration

• Share — click the Link button in the Editor to share a prompt
• Comments — add comments to discuss changes
• Labels — categorize and organize prompts

Parameters reference

1
{
2
  "variables": {
3
    "user_name": "John",
4
    "task": "summarize"
5
  }
6
}

1
{
2
  "prompt": {
3
    "prompt_id": "my-prompt",
4
    "schema_version": 2
5
  },
6
  "fallback_models": ["gpt-4o", "claude-sonnet-4-20250514", "gemini-2.5-flash"]
7
}