Create Evaluation | Humanloop Docs

curl -X POST https://api.humanloop.com/v5/evaluations \
     -H "X-API-KEY: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evaluators": [
    {
      "version_id": "version_id"
    }
  ]
}'

{
  "id": "id",
  "runs_count": 1,
  "evaluators": [
    {
      "version": {
        "path": "path",
        "id": "id",
        "spec": {
          "arguments_type": "target_free",
          "return_type": "boolean",
          "evaluator_type": "llm"
        },
        "name": "name",
        "version_id": "version_id",
        "created_at": "2024-01-15T09:30:00Z",
        "updated_at": "2024-01-15T09:30:00Z",
        "last_used_at": "2024-01-15T09:30:00Z",
        "version_logs_count": 1,
        "total_logs_count": 1,
        "inputs": [
          {
            "name": "name"
          }
        ]
      },
      "orchestrated": true,
      "added_at": "2024-01-15T09:30:00Z"
    }
  ],
  "created_at": "2024-01-15T09:30:00Z",
  "updated_at": "2024-01-15T09:30:00Z",
  "name": "name",
  "file_id": "file_id",
  "created_by": {
    "id": "id",
    "email_address": "email_address",
    "full_name": "full_name"
  },
  "url": "url"
}

Create an Evaluation.

Create a new Evaluation by specifying the File to evaluate, and a name for the Evaluation. You can then add Runs to this Evaluation using the POST /evaluations/{id}/runs endpoint.

curl -X POST https://api.humanloop.com/v5/evaluations \
     -H "X-API-KEY: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evaluators": [
    {
      "version_id": "version_id"
    }
  ]
}'

{
  "id": "id",
  "runs_count": 1,
  "evaluators": [
    {
      "version": {
        "path": "path",
        "id": "id",
        "spec": {
          "arguments_type": "target_free",
          "return_type": "boolean",
          "evaluator_type": "llm"
        },
        "name": "name",
        "version_id": "version_id",
        "created_at": "2024-01-15T09:30:00Z",
        "updated_at": "2024-01-15T09:30:00Z",
        "last_used_at": "2024-01-15T09:30:00Z",
        "version_logs_count": 1,
        "total_logs_count": 1,
        "inputs": [
          {
            "name": "name"
          }
        ]
      },
      "orchestrated": true,
      "added_at": "2024-01-15T09:30:00Z"
    }
  ],
  "created_at": "2024-01-15T09:30:00Z",
  "updated_at": "2024-01-15T09:30:00Z",
  "name": "name",
  "file_id": "file_id",
  "created_by": {
    "id": "id",
    "email_address": "email_address",
    "full_name": "full_name"
  },
  "url": "url"
}

Create an Evaluation.

Create a new Evaluation by specifying the File to evaluate, and a name for the Evaluation. You can then add Runs to this Evaluation using the POST /evaluations/{id}/runs endpoint.

Authentication

X-API-KEYstring

API Key authentication via header

Request

This endpoint expects an object.

evaluatorslist of objectsRequired

The Evaluators used to evaluate.

fileobjectOptional

The File to associate with the Evaluation. This File contains the Logs you're evaluating.

namestringOptional

Name of the Evaluation to help identify it. Must be unique within the associated File.

Response

Successful Response

idstring

Unique identifier for the Evaluation. Starts with evr.

runs_countinteger

The total number of Runs in the Evaluation.

evaluatorslist of objects

The Evaluator Versions used to evaluate.

created_atdatetime

updated_atdatetime

namestring

Name of the Evaluation to help identify it. Must be unique among Evaluations associated with File.

file_idstring

Unique identifier for the File associated with the Evaluation.

created_byany

urlstring

URL to view the Evaluation on the Humanloop.

Errors

422

Create Evaluations Post Request Unprocessable Entity Error

$	curl -X POST https://api.humanloop.com/v5/evaluations \
>	-H "X-API-KEY: <apiKey>" \
>	-H "Content-Type: application/json" \
>	-d '{
>	"evaluators": [
>	{
>	"version_id": "version_id"
>	}
>	]
>	}'

1	{
2	"id": "id",
3	"runs_count": 1,
4	"evaluators": [
5	{
6	"version": {
7	"path": "path",
8	"id": "id",
9	"spec": {
10	"arguments_type": "target_free",
11	"return_type": "boolean",
12	"evaluator_type": "llm"
13	},
14	"name": "name",
15	"version_id": "version_id",
16	"created_at": "2024-01-15T09:30:00Z",
17	"updated_at": "2024-01-15T09:30:00Z",
18	"last_used_at": "2024-01-15T09:30:00Z",
19	"version_logs_count": 1,
20	"total_logs_count": 1,
21	"inputs": [
22	{
23	"name": "name"
24	}
25	]
26	},
27	"orchestrated": true,
28	"added_at": "2024-01-15T09:30:00Z"
29	}
30	],
31	"created_at": "2024-01-15T09:30:00Z",
32	"updated_at": "2024-01-15T09:30:00Z",
33	"name": "name",
34	"file_id": "file_id",
35	"created_by": {
36	"id": "id",
37	"email_address": "email_address",
38	"full_name": "full_name"
39	},
40	"url": "url"
41	}