Update Evaluation Run

1 from humanloop import Humanloop 2 3 client = Humanloop( 4 api_key="YOUR_API_KEY", 5 ) 6 client.evaluations.update_evaluation_run( 7 id="id", 8 run_id="run_id", 9 )

1 { 2 "id": "id", 3 "orchestrated": true, 4 "added_at": "2024-01-15T09:30:00Z", 5 "created_at": "2024-01-15T09:30:00Z", 6 "status": "pending", 7 "control": true, 8 "dataset": { 9 "path": "path", 10 "id": "id", 11 "name": "name", 12 "version_id": "version_id", 13 "created_at": "2024-01-15T09:30:00Z", 14 "updated_at": "2024-01-15T09:30:00Z", 15 "status": "uncommitted", 16 "last_used_at": "2024-01-15T09:30:00Z", 17 "datapoints_count": 1, 18 "directory_id": "directory_id", 19 "description": "description", 20 "readme": "readme", 21 "tags": [ 22 "tags" 23 ], 24 "type": "dataset", 25 "environments": [ 26 { 27 "id": "id", 28 "created_at": "2024-01-15T09:30:00Z", 29 "name": "name", 30 "tag": "default" 31 } 32 ], 33 "created_by": { 34 "id": "id", 35 "email_address": "email_address", 36 "full_name": "full_name" 37 }, 38 "committed_by": { 39 "id": "id", 40 "email_address": "email_address", 41 "full_name": "full_name" 42 }, 43 "committed_at": "2024-01-15T09:30:00Z", 44 "commit_message": "commit_message", 45 "datapoints": [ 46 { 47 "id": "id" 48 } 49 ], 50 "attributes": { 51 "key": "value" 52 } 53 }, 54 "version": { 55 "path": "path", 56 "id": "id", 57 "directory_id": "directory_id", 58 "model": "model", 59 "endpoint": "complete", 60 "template": "template", 61 "template_language": "default", 62 "provider": "openai", 63 "max_tokens": 1, 64 "temperature": 1.1, 65 "top_p": 1.1, 66 "stop": "stop", 67 "presence_penalty": 1.1, 68 "frequency_penalty": 1.1, 69 "other": { 70 "key": "value" 71 }, 72 "seed": 1, 73 "response_format": { 74 "type": "json_object", 75 "json_schema": { 76 "key": "value" 77 } 78 }, 79 "reasoning_effort": "high", 80 "tools": [ 81 { 82 "name": "name", 83 "description": "description" 84 } 85 ], 86 "linked_tools": [ 87 { 88 "name": "name", 89 "description": "description", 90 "id": "id", 91 "version_id": "version_id" 92 } 93 ], 94 "attributes": { 95 "key": "value" 96 }, 97 "commit_message": "commit_message", 98 "description": "description", 99 "tags": [ 100 "tags" 101 ], 102 "readme": "readme", 103 "name": "name", 104 "version_id": "version_id", 105 "type": "prompt", 106 "environments": [ 107 { 108 "id": "id", 109 "created_at": "2024-01-15T09:30:00Z", 110 "name": "name", 111 "tag": "default" 112 } 113 ], 114 "created_at": "2024-01-15T09:30:00Z", 115 "updated_at": "2024-01-15T09:30:00Z", 116 "created_by": { 117 "id": "id", 118 "email_address": "email_address", 119 "full_name": "full_name" 120 }, 121 "committed_by": { 122 "id": "id", 123 "email_address": "email_address", 124 "full_name": "full_name" 125 }, 126 "committed_at": "2024-01-15T09:30:00Z", 127 "status": "uncommitted", 128 "last_used_at": "2024-01-15T09:30:00Z", 129 "version_logs_count": 1, 130 "total_logs_count": 1, 131 "inputs": [ 132 { 133 "name": "name" 134 } 135 ], 136 "evaluator_aggregates": [ 137 { 138 "value": 1.1, 139 "evaluator_id": "evaluator_id", 140 "evaluator_version_id": "evaluator_version_id", 141 "created_at": "2024-01-15T09:30:00Z", 142 "updated_at": "2024-01-15T09:30:00Z" 143 } 144 ] 145 }, 146 "created_by": { 147 "id": "id", 148 "email_address": "email_address", 149 "full_name": "full_name" 150 } 151 }

Update an Evaluation Run.

Specify control=true to use this Run as the control Run for the Evaluation. You can cancel a running/pending Run, or mark a Run that uses external or human Evaluators as completed.

Path parameters

idstringRequired

Unique identifier for Evaluation.

run_idstringRequired

Unique identifier for Run.

Request

This endpoint expects an object.

controlbooleanOptional

If True, this Run will be used as the control in the Evaluation. Stats for other Runs will be compared to this Run. This will replace any existing control Run.

statusenumOptional

Used to set the Run to cancelled or completed. Can only be used if the Run is currently pending or running.

Allowed values:

Response

Successful Response

idstring

Unique identifier for the Run.

orchestratedboolean

Whether the Run is orchestrated by Humanloop.

added_atdatetime

When the Run was added to the Evaluation.

created_atdatetime

When the Run was created.

statusenum

The status of the Run.

Allowed values:

controlboolean

Stats for other Runs will be displayed in comparison to the control Run.

datasetobjectOptional

The Dataset used in the Run.

versionobjectOptional

The version used in the Run.

created_byanyOptional

The User who created the Run.

Path parameters

Headers

Request

Response

Errors