Humanloop API

Get a chat response by providing details of the model configuration in the request.

Simple

Chat

ai-assistant-with-tools

Chat Deployed

Get chat response for a specific model configuration.

Chat Model Config

Create Experiment Stream

Create Experiment

Chats

Create a completion by providing details of the model configuration in the request.

Using ID and session

Create

Completion Deployed

Create a completion for a specific model configuration.

Completion Model Config

Completions

Update

Delete

Datapoints

List For Project

Whether to include evaluator aggregates in the response.

Page size for pagination. Number of projects to fetch.

Case-insensitive filter for project name.

Case-insensitive filter for users in the project. This filter matches against both email address and name of users.

List

Get an array of versions associated to your file.

List Configs

Create Feedback Type

Update Feedback Types

Page size for pagination. Number of logs to export.

Export

Get Active Config

Deactivate Config

Active Config

Get an array of environments with the deployed configs associated to your project.

List Deployed Configs

Deploy Config

Delete Deployed Config

Deployed Config

Projects

List 

Create Datapoint

Datasets

Number of evaluation results to retrieve.

List Datapoints

Result

Update Status

Add evaluators to an existing evaluation run.

Add Evaluators

Only return evaluations created after this date.

Only return evaluations created before this date.

Get Evaluations

Evaluations

Get all evaluators within your organization.

Create an evaluator within your organization.

Get an evaluator within your organization.

Delete an evaluator within your organization.

Update an evaluator within your organization.

Evaluators

Feedback

Log a datapoint or array of datapoints to your Humanloop project.

Update By Reference

Update a logged datapoint in your Humanloop project.

Logs

Register

Export a model config to a .prompt file by ID.

Export by ID

Serialize a model config to a .prompt file format.

Serialize

Deserialize a model config from a .prompt file format.

Deserialize

Model Configs

Sessions

AgentConfigResponse

CategoricalFeedbackLabel

Whether the feedback sentiment is positive or negative.

ChatDataResponse

Unique ID for the model inputs and output logged to Humanloop. Use this when recording feedback later.

Output text returned from the provider model with leading and trailing whitespaces stripped.

Raw output text returned from the provider model.

The model configuration used to create the generation.

Results of any tools run during the generation.

The messages passed to the to provider chat endpoint.

ChatMessageWithToolCall

Tool call that this message is responding to.

A list of tool calls requested by the assistant.

ChatMessageWithToolCallContentItem

ChatResponse

Unique identifier of the parent project. Will not be provided if the request was made without providing a project name or id

The suffix that comes after a completion of inserted text. Useful for completions that act like inserts.

End-user ID passed through to provider call.

Counts of the number of tokens used and related stats.

The raw responses returned by the model provider.

The raw request sent to the model provider.

Controls how the model uses tools. The following options are supported: 'none' forces the model to not call a tool; the default when no tools are provided as part of the model config. 'auto' the model can decide to call one of the provided tools; the default when tools are provided as part of the model config. Providing {'type': 'function', 'function': {name': <TOOL_NAME>}} forces the model to use the named function.

ChatResponseToolChoice

ChatRole

CompletionResponse

How many completions to make for each set of inputs.

Array containing the generation responses.

Config

ConfigResponse

ConfigType

Content

CreateDatapointRequest

The inputs to the prompt template for this datapoint.

Object with criteria necessary to evaluate this datapoint.

CreateDatapointRequestTargetValue

CreateDatapointsByLogsRequest

The IDs of the logs to create datapoints from.

CreateLogResponse

CreateSessionResponse

DashboardConfiguration

DataResponse

The inputs passed to the prompt template.

DatapointResponse

DatapointResponseTargetValue

DatasetResponse

Datasets are now files and do not belong to projects. If this dataset was created before that change, the legacy project ID will be provided here, otherwise an empty string will be returned.

EnvironmentProjectConfigResponse

Name of the project the model config belongs to.

Model config unique identifier generated by Humanloop.

A friendly display name for the model config.

Tag for the environment e.g. 'default' or 'other'.

EnvironmentRequest

EvaluationDatapointSnapshotResponse

EvaluationResponse

EvaluationResultResponse

EvaluationStatus

Enum representing the possible argument types of an evaluator.

EvaluatorArgumentsType

EvaluatorConfigResponse

The model config defining the LLM evaluator.

The code for the evaluator. This code will be executed in a sandboxed environment.

Whether this evaluator is target-free or target-required.

The type of the return value of the evaluator.

EvaluatorResponse

The project where the evaluator logs are stored.

Enum representing the possible return types of an evaluator.

EvaluatorReturnTypeEnum

EvaluatorType

The type of feedback. The default feedback types available are 'rating', 'action', 'issue', 'correction', and 'comment'.

The feedback value to set. This would be the appropriate text for 'correction' or 'comment', or a label to apply for 'rating', 'action', or 'issue'.

ID to associate the feedback to a previously logged datapoint.

A unique identifier to who provided the feedback.

User defined timestamp for when the feedback was created.

FeedbackClass

FeedbackLabelRequest

FeedbackRequest

The feedback value to be set. This field should be left blank when unsetting 'rating', 'correction' or 'comment', but is required otherwise.

If true, the value for this feedback is unset.

FeedbackRequestType

FeedbackRequestValue

FeedbackResponse

FeedbackResponseType

FeedbackResponseValue

FeedbackType

FeedbackTypeModel

FeedbackTypeModelType

FeedbackTypes

FeedbackValue

FileType

A function tool to be called by the model where user owns runtime.

FunctionTool

FunctionToolChoice

GenericConfigResponse

GetModelConfigResponse

Number of datapoints associated with this project model config.

Aggregates of evaluators for the model config.

ID of environment to reference in subsequent log calls.

HTTPValidationError

ImageChatContent

ImageUrl

Either a URL of the image or the base64 encoded image data.

ImageUrlDetail

Judgment

LabelSentiment

LinkedToolRequest

Whether the tool is strict or not. If strict, the model will be forced to respond with JSON matching the parameters schema.

LogRequest

Unique project name. If no project exists with this name, a new project will be created.

ID of the session to associate the datapoint.

ID associated to the parent datapoint in a session.

Identifies where the model was called from.

Whether the request/response payloads will be stored on Humanloop.

ID of the source datapoint if this is a log derived from a datapoint in a dataset.

Unique ID of a config to associate to the log.

The environment name used to create the log.

Optional parameter to provide feedback with your logged datapoint.

User defined timestamp for when the log was created.

Number of tokens in the prompt used to generate the output.

Number of tokens in the output generated by the model.

Cost in dollars associated to the tokens in the prompt.

Cost in dollars associated to the tokens in the output.

LogRequestFeedback

LogResponse

The name of the project associated with this log

The unique ID of the project associated with this log.

Unique user-provided string identifying the datapoint.

1	[
2	{
3	"id": "id",
4	"status": "pending",
5	"config": {
6	"type": "generic",
7	"id": "config",
8	"other": {
9	"key": "value"
10	},
11	"created_by": {
12	"id": "id",
13	"email_address": "email_address",
14	"verified": true
15	},
16	"status": "config",
17	"name": "config",
18	"description": "config"
19	},
20	"created_at": "2024-01-15T09:30:00Z",
21	"updated_at": "2024-01-15T09:30:00Z",
22	"evaluators": [
23	{
24	"name": "name",
25	"description": "description",
26	"arguments_type": "target_free",
27	"return_type": "boolean",
28	"type": "python",
29	"id": "id",
30	"created_at": "2024-01-15T09:30:00Z",
31	"updated_at": "2024-01-15T09:30:00Z"
32	}
33	],
34	"dataset": {
35	"id": "id",
36	"project_id": "project_id",
37	"name": "name",
38	"description": "description",
39	"datapoint_count": 1,
40	"created_at": "2024-01-15T09:30:00Z",
41	"updated_at": "2024-01-15T09:30:00Z"
42	},
43	"dataset_version_id": "dataset_version_id",
44	"dataset_snapshot": {
45	"id": "id",
46	"project_id": "project_id",
47	"name": "name",
48	"description": "description",
49	"datapoint_count": 1,
50	"created_at": "2024-01-15T09:30:00Z",
51	"updated_at": "2024-01-15T09:30:00Z"
52	},
53	"evaluator_aggregates": [
54	{
55	"model_config_id": "model_config_id",
56	"evaluator_id": "evaluator_id",
57	"evaluator_version_id": "evaluator_version_id"
58	}
59	]
60	}
61	]

List For Project

Path parameters

Query parameters

Response

Errors

1	curl https://api.humanloop.com/v4/projects/project_id/evaluations \
2	-H "X-API-KEY: <apiKey>"