Add Evaluators

POST

Add Evaluators to an Evaluation.

The Evaluators will be run on the Logs generated for the Evaluation.

Path parameters

idstringRequired

Unique identifier for Evaluation.

Request

This endpoint expects an object.

evaluatorslist of objectsRequired

The Evaluators to add to this Evaluation.

Response

Successful Response

idstring

Unique identifier for the Evaluation. Starts with evr.

runs_countinteger

The total number of Runs in the Evaluation.

evaluatorslist of objects

The Evaluator Versions used to evaluate.

created_atdatetime

updated_atdatetime

namestringOptional

Name of the Evaluation to help identify it. Must be unique among Evaluations associated with File.

file_idstringOptional

Unique identifier for the File associated with the Evaluation.

created_byanyOptional

urlstringOptional

URL to view the Evaluation on the Humanloop.

Errors

POST

1 curl -X POST https://api.humanloop.com/v5/evaluations/id/evaluators \
2      -H "X-API-KEY: <apiKey>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "evaluators": [
6     {
7       "version_id": "version_id"
8     }
9   ]
10 }'

1 {
2   "id": "id",
3   "runs_count": 1,
4   "evaluators": [
5     {
6       "version": {
7         "path": "path",
8         "id": "id",
9         "spec": {
10           "arguments_type": "target_free",
11           "return_type": "boolean",
12           "evaluator_type": "llm"
13         },
14         "name": "name",
15         "version_id": "version_id",
16         "created_at": "2024-01-15T09:30:00Z",
17         "updated_at": "2024-01-15T09:30:00Z",
18         "status": "uncommitted",
19         "last_used_at": "2024-01-15T09:30:00Z",
20         "version_logs_count": 1,
21         "total_logs_count": 1,
22         "inputs": [
23           {
24             "name": "name"
25           }
26         ]
27       },
28       "orchestrated": true,
29       "added_at": "2024-01-15T09:30:00Z"
30     }
31   ],
32   "created_at": "2024-01-15T09:30:00Z",
33   "updated_at": "2024-01-15T09:30:00Z",
34   "name": "name",
35   "file_id": "file_id",
36   "created_by": {
37     "id": "id",
38     "email_address": "email_address",
39     "full_name": "full_name"
40   },
41   "url": "url"
42 }

Remove Evaluator

Up Next

Built with

1	curl -X POST https://api.humanloop.com/v5/evaluations/id/evaluators \
2	-H "X-API-KEY: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"evaluators": [
6	{
7	"version_id": "version_id"
8	}
9	]
10	}'

1	{
2	"id": "id",
3	"runs_count": 1,
4	"evaluators": [
5	{
6	"version": {
7	"path": "path",
8	"id": "id",
9	"spec": {
10	"arguments_type": "target_free",
11	"return_type": "boolean",
12	"evaluator_type": "llm"
13	},
14	"name": "name",
15	"version_id": "version_id",
16	"created_at": "2024-01-15T09:30:00Z",
17	"updated_at": "2024-01-15T09:30:00Z",
18	"status": "uncommitted",
19	"last_used_at": "2024-01-15T09:30:00Z",
20	"version_logs_count": 1,
21	"total_logs_count": 1,
22	"inputs": [
23	{
24	"name": "name"
25	}
26	]
27	},
28	"orchestrated": true,
29	"added_at": "2024-01-15T09:30:00Z"
30	}
31	],
32	"created_at": "2024-01-15T09:30:00Z",
33	"updated_at": "2024-01-15T09:30:00Z",
34	"name": "name",
35	"file_id": "file_id",
36	"created_by": {
37	"id": "id",
38	"email_address": "email_address",
39	"full_name": "full_name"
40	},
41	"url": "url"
42	}