For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Sign inBook a demo
DocsReferenceChangelog
DocsReferenceChangelog
  • Introduction
    • Overview
    • Errors
  • SDK
    • Overview
    • Decorators
    • Run Evaluation
  • API
      • GETList Evaluations
      • POSTCreate Evaluation
      • POSTAdd Evaluators
      • DELRemove Evaluator
      • GETGet Evaluation
      • DELDelete Evaluation
      • GETList Runs for Evaluation
      • POSTCreate Run
      • POSTAdd Existing Run
      • DELRemove Run
      • PATCHUpdate Evaluation Run
      • POSTAdd Logs to Run
      • GETGet Evaluation Stats
      • GETGet Logs for Evaluation
LogoLogo
Sign inBook a demo
APIEvaluations

Update Evaluation Run

PATCH
https://api.humanloop.com/v5/evaluations/:id/runs/:run_id
PATCH
/v5/evaluations/:id/runs/:run_id
$curl -X PATCH https://api.humanloop.com/v5/evaluations/id/runs/run_id \
>     -H "X-API-KEY: <apiKey>" \
>     -H "Content-Type: application/json" \
>     -d '{}'
1{
2  "id": "id",
3  "orchestrated": true,
4  "added_at": "2024-01-15T09:30:00Z",
5  "created_at": "2024-01-15T09:30:00Z",
6  "status": "pending",
7  "control": true,
8  "dataset": {
9    "path": "path",
10    "id": "id",
11    "name": "name",
12    "version_id": "version_id",
13    "created_at": "2024-01-15T09:30:00Z",
14    "updated_at": "2024-01-15T09:30:00Z",
15    "last_used_at": "2024-01-15T09:30:00Z",
16    "datapoints_count": 1,
17    "directory_id": "directory_id",
18    "description": "description",
19    "schema": {
20      "key": "value"
21    },
22    "readme": "readme",
23    "tags": [
24      "tags"
25    ],
26    "type": "dataset",
27    "environments": [
28      {
29        "id": "id",
30        "created_at": "2024-01-15T09:30:00Z",
31        "name": "name",
32        "tag": "default"
33      }
34    ],
35    "created_by": {
36      "id": "id",
37      "email_address": "email_address",
38      "full_name": "full_name"
39    },
40    "version_name": "version_name",
41    "version_description": "version_description",
42    "datapoints": [
43      {
44        "id": "id"
45      }
46    ],
47    "attributes": {
48      "key": "value"
49    }
50  },
51  "version": {
52    "path": "path",
53    "id": "id",
54    "directory_id": "directory_id",
55    "model": "model",
56    "endpoint": "complete",
57    "template": "template",
58    "template_language": "default",
59    "provider": "anthropic",
60    "max_tokens": 1,
61    "temperature": 1.1,
62    "top_p": 1.1,
63    "stop": "stop",
64    "presence_penalty": 1.1,
65    "frequency_penalty": 1.1,
66    "other": {
67      "key": "value"
68    },
69    "seed": 1,
70    "response_format": {
71      "type": "json_object",
72      "json_schema": {
73        "key": "value"
74      }
75    },
76    "reasoning_effort": "high",
77    "tools": [
78      {
79        "name": "name",
80        "description": "description"
81      }
82    ],
83    "linked_tools": [
84      {
85        "name": "name",
86        "description": "description",
87        "id": "id",
88        "version_id": "version_id"
89      }
90    ],
91    "attributes": {
92      "key": "value"
93    },
94    "version_name": "version_name",
95    "version_description": "version_description",
96    "description": "description",
97    "tags": [
98      "tags"
99    ],
100    "readme": "readme",
101    "name": "name",
102    "schema": {
103      "key": "value"
104    },
105    "version_id": "version_id",
106    "type": "prompt",
107    "environments": [
108      {
109        "id": "id",
110        "created_at": "2024-01-15T09:30:00Z",
111        "name": "name",
112        "tag": "default"
113      }
114    ],
115    "created_at": "2024-01-15T09:30:00Z",
116    "updated_at": "2024-01-15T09:30:00Z",
117    "created_by": {
118      "id": "id",
119      "email_address": "email_address",
120      "full_name": "full_name"
121    },
122    "last_used_at": "2024-01-15T09:30:00Z",
123    "version_logs_count": 1,
124    "total_logs_count": 1,
125    "inputs": [
126      {
127        "name": "name"
128      }
129    ],
130    "evaluator_aggregates": [
131      {
132        "value": 1.1,
133        "evaluator_id": "evaluator_id",
134        "evaluator_version_id": "evaluator_version_id",
135        "created_at": "2024-01-15T09:30:00Z",
136        "updated_at": "2024-01-15T09:30:00Z"
137      }
138    ],
139    "raw_file_content": "raw_file_content"
140  },
141  "created_by": {
142    "id": "id",
143    "email_address": "email_address",
144    "full_name": "full_name"
145  }
146}

Update an Evaluation Run.

Specify control=true to use this Run as the control Run for the Evaluation. You can cancel a running/pending Run, or mark a Run that uses external or human Evaluators as completed.

Was this page helpful?
Previous

Add Logs to Run

Next
Built with

Authentication

X-API-KEYstring
API Key authentication via header

Path parameters

idstringRequired
Unique identifier for Evaluation.
run_idstringRequired
Unique identifier for Run.

Request

This endpoint expects an object.
controlbooleanOptional

If True, this Run will be used as the control in the Evaluation. Stats for other Runs will be compared to this Run. This will replace any existing control Run.

statusenumOptional

Used to set the Run to cancelled or completed. Can only be used if the Run is currently pending or running.

Allowed values:

Response

Successful Response
idstring
Unique identifier for the Run.
orchestratedboolean
Whether the Run is orchestrated by Humanloop.
added_atdatetime
When the Run was added to the Evaluation.
created_atdatetime
When the Run was created.
statusenum
The status of the Run.
Allowed values:
controlboolean
Stats for other Runs will be displayed in comparison to the control Run.
datasetobject
The Dataset used in the Run.
versionobject
The version used in the Run.
created_byany
The User who created the Run.

Errors

422
Update Evaluation Run Evaluations ID Runs Run ID Patch Request Unprocessable Entity Error