API ReferenceEvaluators

Create evaluator version

Commit the current draft and create the next draft version. Supplying only version_description commits the existing draft snapshot; supplying configuration fields commits with changes.

Path parameters

evaluator_idstringRequired

Evaluator ID. To run a specific version, pass an ID with a version suffix where supported, for example evl_abc123:2.

Headers

AuthorizationstringRequired

Bearer token. Use Bearer YOUR_API_KEY for API key auth or Bearer <JWT> for dashboard auth.

Request

Create a new evaluator version. If only version_description is provided, the backend commits the current draft and copies its configuration.

namestringOptional
evaluator_slugstringOptional

Organization-scoped evaluator slug.

typeenumOptional
Allowed values:
score_value_typeenumOptional
eval_classenumOptional

Optional pre-built evaluator template.

Allowed values:
descriptionstring or nullOptional
score_configobjectOptional

Score configuration. For numerical/percentage scores, use min_score and max_score. For select scores, use choices.

passing_conditionsobject or nullOptional
Passing conditions in the standard Respan filter format.
llm_configobject or nullOptional
LLM grader configuration. The backend validates this against the selected evaluator form.
code_configobject or nullOptional
Code grader configuration.
configurationsmap from strings to anyOptional

Legacy user-facing configuration object. New clients should prefer llm_config, code_config, score_config, and passing_conditions.

categorical_choiceslist of objectsOptional
starredbooleanOptional
version_descriptionstringOptional
Commit message for this version.

Response

Created evaluator version.
idstring
Stable evaluator ID shared by all versions.
version_idstring
Unique ID for this evaluator version.
versioninteger
Version number. The first draft is version 0.
is_read_onlyboolean
Whether this version is committed and immutable.
namestring
typeenum
Allowed values:
score_value_typeenum
version_descriptionstring
Commit message for this version.
descriptionstring or null
score_configobject

Score configuration. For numerical/percentage scores, use min_score and max_score. For select scores, use choices.

passing_conditionsobject or null
Passing conditions in the standard Respan filter format.
llm_configobject or null
LLM grader configuration. The backend validates this against the selected evaluator form.
code_configobject or null
Code grader configuration.
configurationsmap from strings to any

Legacy user-facing configuration object. New clients should prefer llm_config, code_config, score_config, and passing_conditions.

categorical_choiceslist of objects or null
eval_classenum

Optional pre-built evaluator template.

Allowed values:
starredboolean
created_atdatetime
updated_atdatetime
created_byobject
updated_byobject

Errors

400
Bad Request Error
401
Unauthorized Error
404
Not Found Error