Get Evaluations

GET

Get the evaluations associated with a project.

Sorting and filtering are supported through query params for categorical columns and the created_at timestamp.

Sorting is supported for the dataset, config, status and evaluator-{evaluator_id} columns. Specify sorting with the sort query param, with values {column}.{ordering}. E.g. ?sort=dataset.asc&sort=status.desc will yield a multi-column sort. First by dataset then by status.

Filtering is supported for the id, dataset, config and status columns.

Specify filtering with the id_filter, dataset_filter, config_filter and status_filter query params.

E.g. ?dataset_filter=my_dataset&dataset_filter=my_other_dataset&status_filter=running will only show rows where the dataset is “my_dataset” or “my_other_dataset”, and where the status is “running”.

An additional date range filter is supported for the created_at column. Use the start_date and end_date query parameters to configure this.

Query parameters

project_idstringRequired

String ID of project. Starts with pr_.

idstringOptional

A list of evaluation run ids to filter on. Starts with ev_.

start_datestringOptional

Only return evaluations created after this date.

end_datestringOptional

Only return evaluations created before this date.

sizeintegerOptional

pageintegerOptional

evaluatee_idstringOptional

String ID of evaluatee version to return. If not defined, the first evaluatee will be returned. Starts with evv_.

Response

Successful Response

recordslist of objects

pageinteger

sizeinteger

totalinteger

Errors

GET

1 curl -G https://api.humanloop.com/v4/evaluations \
2      -H "X-API-KEY: <apiKey>" \
3      -d project_id=project_id

1 {
2   "records": [
3     {
4       "id": "id",
5       "status": "pending",
6       "config": {
7         "type": "generic",
8         "id": "config",
9         "name": "config",
10         "status": "config"
11       },
12       "created_at": "2024-01-15T09:30:00Z",
13       "updated_at": "2024-01-15T09:30:00Z",
14       "evaluators": [
15         {
16           "name": "name",
17           "description": "description",
18           "arguments_type": "target_free",
19           "return_type": "boolean",
20           "type": "python",
21           "id": "id",
22           "created_at": "2024-01-15T09:30:00Z",
23           "updated_at": "2024-01-15T09:30:00Z"
24         }
25       ],
26       "dataset": {
27         "id": "id",
28         "name": "name",
29         "datapoint_count": 1,
30         "created_at": "2024-01-15T09:30:00Z",
31         "updated_at": "2024-01-15T09:30:00Z",
32         "project_id": "project_id"
33       },
34       "dataset_version_id": "dataset_version_id",
35       "dataset_snapshot": {
36         "id": "id",
37         "name": "name",
38         "datapoint_count": 1,
39         "created_at": "2024-01-15T09:30:00Z",
40         "updated_at": "2024-01-15T09:30:00Z",
41         "project_id": "project_id"
42       },
43       "evaluator_aggregates": [
44         {
45           "model_config_id": "model_config_id",
46           "evaluator_id": "evaluator_id",
47           "evaluator_version_id": "evaluator_version_id"
48         }
49       ]
50     }
51   ],
52   "page": 1,
53   "size": 1,
54   "total": 1
55 }

List

Up Next

Built with

1	curl -G https://api.humanloop.com/v4/evaluations \
2	-H "X-API-KEY: <apiKey>" \
3	-d project_id=project_id

1	{
2	"records": [
3	{
4	"id": "id",
5	"status": "pending",
6	"config": {
7	"type": "generic",
8	"id": "config",
9	"name": "config",
10	"status": "config"
11	},
12	"created_at": "2024-01-15T09:30:00Z",
13	"updated_at": "2024-01-15T09:30:00Z",
14	"evaluators": [
15	{
16	"name": "name",
17	"description": "description",
18	"arguments_type": "target_free",
19	"return_type": "boolean",
20	"type": "python",
21	"id": "id",
22	"created_at": "2024-01-15T09:30:00Z",
23	"updated_at": "2024-01-15T09:30:00Z"
24	}
25	],
26	"dataset": {
27	"id": "id",
28	"name": "name",
29	"datapoint_count": 1,
30	"created_at": "2024-01-15T09:30:00Z",
31	"updated_at": "2024-01-15T09:30:00Z",
32	"project_id": "project_id"
33	},
34	"dataset_version_id": "dataset_version_id",
35	"dataset_snapshot": {
36	"id": "id",
37	"name": "name",
38	"datapoint_count": 1,
39	"created_at": "2024-01-15T09:30:00Z",
40	"updated_at": "2024-01-15T09:30:00Z",
41	"project_id": "project_id"
42	},
43	"evaluator_aggregates": [
44	{
45	"model_config_id": "model_config_id",
46	"evaluator_id": "evaluator_id",
47	"evaluator_version_id": "evaluator_version_id"
48	}
49	]
50	}
51	],
52	"page": 1,
53	"size": 1,
54	"total": 1
55	}