For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
DiscordPlatform
DocumentationIntegrationsAPI referenceSDKsChangelog
DocumentationIntegrationsAPI referenceSDKsChangelog
  • API Reference
      • POSTCreate an evaluator
      • POSTList evaluators with filters
      • GETRetrieve an evaluator
      • PATCHUpdate an evaluator
      • DELDelete an evaluator
      • POSTRun an evaluator
      • GETList evaluators
      • GETGet evaluators summary
      • POSTGet filtered evaluators summary
      • PUTReplace an evaluator
      • GETList evaluator versions
      • POSTCreate evaluator version
      • GETRetrieve evaluator version
      • PUTReplace evaluator version
      • PATCHUpdate evaluator version
  • Reference
    • Filters API Reference
LogoLogo
DiscordPlatform
API ReferenceEvaluators

Retrieve an evaluator

GET
https://api.respan.ai/api/evaluators/:evaluator_id/
GET
/api/evaluators/:evaluator_id/
$curl https://api.respan.ai/api/evaluators/evaluator_id/ \
>     -H "Authorization: Bearer sk_live_xxxxx" \
>     -H "Content-Type: application/json"
1{
2  "id": "evl_550e8400",
3  "name": "Response Quality",
4  "type": "llm",
5  "score_value_type": "numerical",
6  "version_id": "evlv_550e8400",
7  "version": 1,
8  "is_read_only": false,
9  "version_description": "Tightened rubric wording",
10  "evaluator_slug": "response_quality",
11  "eval_class": "keywordsai_custom_llm",
12  "description": "Grades whether the response is accurate and complete.",
13  "score_config": {
14    "min_score": 1,
15    "max_score": 5,
16    "choices": [
17      {
18        "name": "Excellent",
19        "value": "excellent"
20      }
21    ]
22  },
23  "passing_conditions": {
24    "primary_score": {
25      "operator": "gte",
26      "value": 3
27    }
28  },
29  "llm_config": {
30    "model": "gpt-4o-mini",
31    "evaluator_definition": "Rate the response quality.\n<input>{{input}}</input>\n<output>{{output}}</output>",
32    "scoring_rubric": "1=Poor, 5=Excellent",
33    "temperature": 0.1,
34    "max_tokens": 200,
35    "top_p": 0.9,
36    "frequency_penalty": 0.2,
37    "presence_penalty": 0.3,
38    "stop": [
39      "\n"
40    ],
41    "response_format": {},
42    "tools": [
43      {}
44    ],
45    "verbosity": "detailed"
46  },
47  "code_config": {
48    "eval_code_snippet": "def main(eval_inputs):\n    return 1 if eval_inputs.get('output') else 0"
49  },
50  "configurations": {},
51  "categorical_choices": [
52    {
53      "name": "Excellent",
54      "value": "excellent"
55    }
56  ],
57  "starred": false,
58  "created_at": "2024-01-15T09:30:00Z",
59  "updated_at": "2024-01-15T09:30:00Z",
60  "created_by": {
61    "id": 101,
62    "first_name": "Alice",
63    "last_name": "Johnson",
64    "email": "alice.johnson@respan.ai"
65  },
66  "updated_by": {
67    "id": 102,
68    "first_name": "Bob",
69    "last_name": "Smith",
70    "email": "bob.smith@respan.ai"
71  },
72  "editor": {
73    "id": 103,
74    "first_name": "Carol",
75    "last_name": "Davis",
76    "email": "carol.davis@respan.ai"
77  },
78  "tags": [
79    {
80      "id": 10,
81      "name": "quality",
82      "color": "#4CAF50"
83    }
84  ]
85}

Retrieve the current draft/latest version of an evaluator by ID.

Was this page helpful?
Previous

Update an evaluator

Next
Built with

Path parameters

evaluator_idstringRequired

Evaluator ID. To run a specific version, pass an ID with a version suffix where supported, for example evl_abc123:2.

Headers

AuthorizationstringRequired

Bearer token. Use Bearer YOUR_API_KEY for API key auth or Bearer <JWT> for dashboard auth.

Response

Evaluator details.
idstring
Stable evaluator ID shared by all versions.
namestring
typeenum
Allowed values:
score_value_typeenum
version_idstring
Unique ID for this evaluator version.
versioninteger
Version number. The first draft is version 0.
is_read_onlyboolean
Whether this version is committed and immutable.
version_descriptionstring
Commit message for this version.
evaluator_slugstring

Organization-scoped evaluator slug.

eval_classenum

Optional pre-built evaluator template.

Allowed values:
descriptionstring or null
score_configobject

Score configuration. For numerical/percentage scores, use min_score and max_score. For select scores, use choices.

passing_conditionsobject or null
Passing conditions in the standard Respan filter format.
llm_configobject or null
LLM grader configuration. The backend validates this against the selected evaluator form.
code_configobject or null
Code grader configuration.
configurationsmap from strings to any

Legacy user-facing configuration object. New clients should prefer llm_config, code_config, score_config, and passing_conditions.

categorical_choiceslist of objects or null
starredboolean
created_atdatetime
updated_atdatetime
created_byobject
updated_byobject
editorobject
tagslist of objects

Errors

401
Unauthorized Error
404
Not Found Error