Humanloop APIChats

Chat

Get a chat response by providing details of the model configuration in the request.

Headers

X-API-KEYstringRequired

Request

This endpoint expects an object.
streamfalseRequired

If true, tokens will be sent as data-only server-sent events. If num_samples > 1, samples are streamed back independently.

messageslist of objectsRequired
The messages passed to the to provider chat endpoint.
model_configobjectRequired
The model configuration used to create a chat response.
projectstringOptional
Unique project name. If no project exists with this name, a new project will be created.
project_idstringOptional

Unique ID of a project to associate to the log. Either this or project must be provided.

session_idstringOptional
ID of the session to associate the datapoint.
session_reference_idstringOptional

A unique string identifying the session to associate the datapoint to. Allows you to log multiple datapoints to a session (using an ID kept by your internal systems) by passing the same session_reference_id in subsequent log requests. Specify at most one of this or session_id.

parent_idstringOptional
ID associated to the parent datapoint in a session.
parent_reference_idstringOptional

A unique string identifying the previously-logged parent datapoint in a session. Allows you to log nested datapoints with your internal system IDs by passing the same reference ID as parent_id in a prior log request. Specify at most one of this or parent_id. Note that this cannot refer to a datapoint being logged in the same request.

inputsmap from strings to anyOptional
The inputs passed to the prompt template.
sourcestringOptional
Identifies where the model was called from.
metadatamap from strings to anyOptional
Any additional metadata to record.
savebooleanOptionalDefaults to true

Whether the request/response payloads will be stored on Humanloop.

source_datapoint_idstringOptional
ID of the source datapoint if this is a log derived from a datapoint in a dataset.
provider_api_keysobjectOptional
API keys required by each provider to make API calls. The API keys provided here are not stored by Humanloop. If not specified here, Humanloop will fall back to the key saved to your organization.
num_samplesintegerOptionalDefaults to 1
The number of generations.
template_languageenumOptional
The template language to use for rendering the template.
Allowed values:
userstringOptional

End-user ID passed through to provider call.

return_inputsbooleanOptionalDefaults to true
Whether to return the inputs in the response. If false, the response will contain an empty dictionary under inputs. This is useful for reducing the size of the response. Defaults to true.
tool_choice"none" or "auto" or "required" or objectOptional

Controls how the model uses tools. The following options are supported: ‘none’ forces the model to not call a tool; the default when no tools are provided as part of the model config. ‘auto’ the model can decide to call one of the provided tools; the default when tools are provided as part of the model config. Providing {‘type’: ‘function’, ‘function’: {name’: <TOOL_NAME>}} forces the model to use the named function.

response_formatobjectOptional

The format of the response. Only type json_object is currently supported for chat.

reasoning_effortenum or integerOptional

Guidance on how many reasoning tokens it should generate before creating a response to the prompt. OpenAI reasoning models (o1, o3-mini) expect a OpenAIReasoningEffort enum. Anthropic reasoning models expect an integer, which signifies the maximum token budget.

seedintegerOptionalDeprecated

Deprecated field: the seed is instead set as part of the request.config object.

tool_callstring or map from strings to stringsOptionalDeprecated

NB: Deprecated with new tool_choice. Controls how the model uses tools. The following options are supported: ‘none’ forces the model to not call a tool; the default when no tools are provided as part of the model config. ‘auto’ the model can decide to call one of the provided tools; the default when tools are provided as part of the model config. Providing {‘name’: <TOOL_NAME>} forces the model to use the provided tool of the same name.

Response

datalist of objects
Array containing the chat responses.
provider_responseslist of any
The raw responses returned by the model provider.
project_idstring or null
Unique identifier of the parent project. Will not be provided if the request was made without providing a project name or id
num_samplesinteger or nullDefaults to 1
The number of chat responses.
logprobsinteger or null

Include the log probabilities of the top n tokens in the provider_response

suffixstring or null
The suffix that comes after a completion of inserted text. Useful for completions that act like inserts.
userstring or null

End-user ID passed through to provider call.

usageobject or null
Counts of the number of tokens used and related stats.
metadatamap from strings to any or null
Any additional metadata to record.
provider_requestmap from strings to any or null
The raw request sent to the model provider.
session_idstring or null
ID of the session if it belongs to one.
tool_choice"none" or "auto" or "required" or object or null

Controls how the model uses tools. The following options are supported: ‘none’ forces the model to not call a tool; the default when no tools are provided as part of the model config. ‘auto’ the model can decide to call one of the provided tools; the default when tools are provided as part of the model config. Providing {‘type’: ‘function’, ‘function’: {name’: <TOOL_NAME>}} forces the model to use the named function.

Errors