> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://respan.ai/docs/_mcp/server.

# Ragas (tracing)

> Trace Ragas metrics, dataset evaluations, experiments, and experiment rows as canonical Respan task spans.

[Ragas](https://docs.ragas.io/) is a framework for evaluating LLM and RAG applications. `respan-instrumentation-ragas` traces Ragas 0.4 metrics, dataset evaluations, and experiments as canonical Respan task spans, preserving the active workflow hierarchy around them.

The adapter covers both the preferred Ragas 0.4 metric collections API and the legacy evaluation surfaces. Nested internal calls are collapsed so one logical metric operation does not produce duplicate sync and async spans.

#### Set up Respan

Create an account at [platform.respan.ai](https://platform.respan.ai) and grab an [API key](https://platform.respan.ai/platform/gateway/api-keys).

Run `npx @respan/cli setup` to set up with your coding agent.

#### Example projects

* [Python package README](https://github.com/RespanAI/respan/tree/main/python-sdks/instrumentations/respan-instrumentation-ragas)
* [Runnable Ragas examples](https://github.com/respanai/respan-example-projects/tree/main/python/tracing/ragas)

## Setup

#### Install packages

```bash
pip install respan-ai respan-instrumentation-ragas
```

#### Set the Respan API key

```bash
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
```

Metric-specific provider credentials are still required for metrics that call an LLM or embedding model. Deterministic metrics such as exact match do not require a provider key.

#### Initialize and score a sample

```python
from ragas.metrics.collections import ExactMatch, StringPresence
from respan import Respan
from respan_instrumentation_ragas import RagasInstrumentor

respan = Respan(
    app_name="ragas-answer-evaluation",
    instrumentations=[RagasInstrumentor()],
)

try:
    exact_match = ExactMatch().score(
        reference="Paris",
        response="Paris",
    )
    string_presence = StringPresence().score(
        reference="Paris",
        response="Paris, France",
    )
    print(exact_match.value, string_presence.value)
finally:
    respan.shutdown()
```

#### View your trace

Open the [Traces page](https://platform.respan.ai/platform/traces) to inspect one task span per metric with its inputs, score, timing, and status.

## Supported operations

| Ragas surface       | Traced operation                                     |
| ------------------- | ---------------------------------------------------- |
| Collection metrics  | `score`, `ascore`, `batch_score`, and `abatch_score` |
| Metric base classes | Single-turn and multi-turn sync and async scoring    |
| Dataset evaluation  | `evaluate` and `aevaluate`                           |
| Experiments         | Experiment `arun` plus each experiment row           |

All of these operations emit canonical Respan `task` spans. Successful spans include `status_code=200`. Failures are re-raised unchanged after recording OpenTelemetry error status, an exception event, backend-visible `status_code`, and `error.message`.

## Async metrics

```python
from ragas.metrics.collections import ExactMatch


async def score_answer() -> float:
    result = await ExactMatch().ascore(
        reference="Paris",
        response="Paris",
    )
    return result.value
```

Batch metric methods are traced as one logical operation, without an extra span for each internal sync-to-async delegation.

## Dataset evaluation

The legacy `evaluate` and `aevaluate` entry points remain supported:

```python
from ragas import EvaluationDataset, evaluate
from ragas.metrics import ExactMatch, StringPresence

dataset = EvaluationDataset.from_list(
    [
        {"response": "Paris", "reference": "Paris"},
        {"response": "Lyon", "reference": "Paris"},
    ]
)
result = evaluate(
    dataset,
    metrics=[ExactMatch(), StringPresence()],
    experiment_name="capital_answers",
    show_progress=False,
)
print(result.to_pandas())
```

## Experiments and rows

Ragas experiments produce a task for the experiment run and a child task for every evaluated row:

```python
from ragas import Dataset, experiment
from ragas.metrics.collections import ExactMatch


@experiment()
async def score_row(row: dict) -> dict:
    score = await ExactMatch().ascore(
        reference=row["reference"],
        response=row["response"],
    )
    return {**row, "exact_match": score.value}


async def run_experiment(dataset: Dataset) -> object:
    return await score_row.arun(dataset, name="answer_quality")
```

## Configuration

| Parameter             | Type           | Default | Description                                                            |
| --------------------- | -------------- | ------- | ---------------------------------------------------------------------- |
| `capture_content`     | `bool`         | `True`  | Capture evaluation inputs, references, responses, scores, and outputs. |
| `instrumentations`    | `list`         | `[]`    | Pass `[RagasInstrumentor()]` to activate Ragas tracing.                |
| `api_key`             | `str \| None`  | `None`  | Falls back to `RESPAN_API_KEY`.                                        |
| `base_url`            | `str \| None`  | `None`  | Falls back to `RESPAN_BASE_URL`.                                       |
| `customer_identifier` | `str \| None`  | `None`  | Default customer identifier for all emitted spans.                     |
| `metadata`            | `dict \| None` | `None`  | Default metadata attached to all emitted spans.                        |

### Disable content capture

```python
from respan import Respan
from respan_instrumentation_ragas import RagasInstrumentor

respan = Respan(
    instrumentations=[RagasInstrumentor(capture_content=False)],
)
```

With capture disabled, evaluation inputs and outputs are omitted. Operation names, duration, lifecycle, status, and exception type remain visible.

## Group evaluations in a workflow

Use a Respan workflow or propagated attributes to group several Ragas operations into one trace:

```python
from respan import Respan, workflow


@workflow(name="ragas_regression_suite")
def run_regression_suite() -> dict[str, float]:
    score = ExactMatch().score(reference="Paris", response="Paris")
    return {"exact_match": score.value}


with Respan.propagate_attributes(
    trace_group_identifier="ragas_regression_suite",
    metadata={"dataset": "capital-cities-v2"},
):
    results = run_regression_suite()
```