List

1 import requests 2 3 url = "https://api.humanloop.com/v4/evaluators" 4 5 headers = {"X-API-KEY": "<apiKey>"} 6 7 response = requests.get(url, headers=headers) 8 9 print(response.json())

1 [ 2 { 3 "name": "name", 4 "description": "description", 5 "arguments_type": "target_free", 6 "return_type": "boolean", 7 "type": "python", 8 "id": "id", 9 "created_at": "2024-01-15T09:30:00Z", 10 "updated_at": "2024-01-15T09:30:00Z", 11 "code": "code", 12 "model_config": { 13 "id": "id", 14 "model": "model", 15 "other": { 16 "key": "value" 17 }, 18 "name": "name", 19 "description": "description", 20 "provider": "openai", 21 "max_tokens": 1, 22 "temperature": 1.1, 23 "top_p": 1.1, 24 "stop": "stop", 25 "presence_penalty": 1.1, 26 "frequency_penalty": 1.1, 27 "seed": 1, 28 "response_format": { 29 "type": "json_object" 30 }, 31 "reasoning_effort": "high", 32 "template_language": "default", 33 "prompt_template": "prompt_template", 34 "chat_template": [ 35 { 36 "role": "user" 37 } 38 ], 39 "tools": [ 40 { 41 "id": "id", 42 "name": "name" 43 } 44 ], 45 "endpoint": "complete", 46 "tool_configs": [ 47 { 48 "id": "id", 49 "status": "status", 50 "name": "name" 51 } 52 ] 53 }, 54 "logging_project": { 55 "id": "id", 56 "name": "name", 57 "users": [ 58 { 59 "id": "id", 60 "email_address": "email_address" 61 } 62 ], 63 "data_count": 1, 64 "feedback_types": [ 65 { 66 "type": "rating" 67 } 68 ], 69 "team_id": "team_id", 70 "created_at": "2024-01-15T09:30:00Z", 71 "updated_at": "2024-01-15T09:30:00Z", 72 "active_config": { 73 "project_id": "project_id", 74 "project_name": "project_name", 75 "created_at": "2024-01-15T09:30:00Z", 76 "updated_at": "2024-01-15T09:30:00Z", 77 "last_used": "2024-01-15T09:30:00Z", 78 "config": { 79 "type": "model", 80 "id": "id", 81 "model": "model" 82 } 83 }, 84 "config_type": "generic", 85 "directory_id": "directory_id" 86 } 87 } 88 ]

Get all evaluators within your organization.

Response

Successful Response

namestring>=1 character<=255 characters

The name of the evaluator.

descriptionstring<=255 characters

The description of the evaluator.

arguments_typeenum

Whether this evaluator is target-free or target-required.

Allowed values:

return_typeenum

The type of the return value of the evaluator.

Allowed values:

typeenum

The type of the evaluator.

Allowed values:

idstring

Unique ID for the evaluator. Starts with evfn_.

created_atdatetime

updated_atdatetime

codestringOptional

The code for the evaluator. This code will be executed in a sandboxed environment.

model_configobjectOptional

The model config defining the LLM evaluator.

logging_projectobjectOptional

The project where the evaluator logs are stored.

Headers

Response

Errors