Get Evaluation Stats

GET

Get Evaluation Stats.

Retrieve aggregate stats for the specified Evaluation. This includes the number of generated Logs for each evaluated version and the corresponding Evaluator statistics (such as the mean and percentiles).

Path parameters

idstringRequired

Unique identifier for Evaluation.

Response

Successful Response

overall_statsobject

Stats for the Evaluation Report as a whole.

version_statslist of objects

Stats for each Evaluated Version in the Evaluation Report.

statusenum

Allowed values: pendingrunningcompletedcancelled

The current status of the Evaluation.

progressstringOptional

A summary string report of the Evaluation’s progress you can print to the command line;helpful when integrating Evaluations with CI/CD.

reportstringOptional

A summary string report of the Evaluation you can print to command line;helpful when integrating Evaluations with CI/CD.

Errors

GET

1 curl https://api.humanloop.com/v5/evaluations/id/stats \
2      -H "X-API-KEY: <apiKey>"

1 {
2   "overall_stats": {
3     "num_datapoints": 1,
4     "total_logs": 1,
5     "total_evaluator_logs": 1
6   },
7   "version_stats": [
8     {
9       "version_id": "version_id",
10       "num_logs": 1,
11       "evaluator_version_stats": [
12         {
13           "evaluator_version_id": "evaluator_version_id",
14           "total_logs": 1,
15           "num_judgments": 1,
16           "num_nulls": 1,
17           "num_errors": 1,
18           "mean": 0,
19           "std": 1,
20           "percentiles": {
21             "0": -2.5,
22             "25": -0.6745,
23             "50": 0,
24             "75": 0.6745,
25             "100": 2.5
26           }
27         }
28       ],
29       "batch_id": "batch_id"
30     }
31   ],
32   "status": "pending",
33   "progress": "progress",
34   "report": "report"
35 }

Get Logs for an Evaluation

Up Next

Built with

1	curl https://api.humanloop.com/v5/evaluations/id/stats \
2	-H "X-API-KEY: <apiKey>"

1	{
2	"overall_stats": {
3	"num_datapoints": 1,
4	"total_logs": 1,
5	"total_evaluator_logs": 1
6	},
7	"version_stats": [
8	{
9	"version_id": "version_id",
10	"num_logs": 1,
11	"evaluator_version_stats": [
12	{
13	"evaluator_version_id": "evaluator_version_id",
14	"total_logs": 1,
15	"num_judgments": 1,
16	"num_nulls": 1,
17	"num_errors": 1,
18	"mean": 0,
19	"std": 1,
20	"percentiles": {
21	"0": -2.5,
22	"25": -0.6745,
23	"50": 0,
24	"75": 0.6745,
25	"100": 2.5
26	}
27	}
28	],
29	"batch_id": "batch_id"
30	}
31	],
32	"status": "pending",
33	"progress": "progress",
34	"report": "report"
35	}