Upsert Dataset

POST

Create a Dataset or update it with a new version if it already exists.

Datasets are identified by the ID or their path. The datapoints determine the versions of the Dataset.

By default, the new Dataset version will be set to the list of Datapoints provided in the request. You can also create a new version by adding or removing Datapoints from an existing version by specifying action as add or remove respectively. In this case, you may specify the version_id or environment query parameters to identify the existing version to base the new version on. If neither is provided, the default deployed version will be used.

If you provide a commit message, then the new version will be committed; otherwise it will be uncommitted. If you try to commit an already committed version, an exception will be raised.

Humanloop also deduplicates Datapoints. If you try to add a Datapoint that already exists, it will be ignored. If you intentionally want to add a duplicate Datapoint, you can add a unique identifier to the Datapoint’s inputs such as {_dedupe_id: <unique ID>}.

Query parameters

version_idstringOptional

ID of the specific Dataset version to base the created Version on. Only used when action is "add" or "remove".

environmentstringOptional

Name of the Environment identifying a deployed Version to base the created Version on. Only used when action is "add" or "remove".

Request

This endpoint expects an object.
datapointslist of objectsRequired

The Datapoints to create this Dataset version with. Modify the action field to determine how these Datapoints are used.

pathstringOptional

Path of the Dataset, including the name. This locates the Dataset in the Humanloop filesystem and is used as as a unique identifier. For example: folder/name or just name.

idstringOptional

ID for an existing Dataset.

actionenumOptional
Allowed values: setaddremove

The action to take with the provided Datapoints.

  • If "set", the created version will only contain the Datapoints provided in this request.
  • If "add", the created version will contain the Datapoints provided in this request in addition to the Datapoints in the target version.
  • If "remove", the created version will contain the Datapoints in the target version except for the Datapoints provided in this request.

If "add" or "remove", one of the version_id or environment query parameters may be provided.

attributesmap from strings to anyOptional

Additional fields to describe the Dataset. Helpful to separate Dataset versions from each other with details on how they were created or used.

commit_messagestringOptional

Message describing the changes made. If provided, a committed version of the Dataset is created. Otherwise, an uncommitted version is created.

Response

Successful Response

pathstring

Path of the Dataset, including the name, which is used as a unique identifier.

idstring

Unique identifier for the Dataset. Starts with ds_.

namestring

Name of the Dataset, which is used as a unique identifier.

version_idstring

Unique identifier for the specific Dataset Version. If no query params provided, the default deployed Dataset Version is returned. Starts with dsv_.

created_atdatetime
updated_atdatetime
statusenum
Allowed values: uncommittedcommitteddeleted

The status of the Dataset Version.

last_used_atdatetime
datapoints_countinteger

The number of Datapoints in this Dataset version.

directory_idstringOptional

ID of the directory that the file is in on Humanloop.

type"dataset"OptionalDefaults to dataset
environmentslist of objectsOptional

The list of environments the Dataset Version is deployed to.

created_byanyOptional

The user who created the Dataset.

committed_byanyOptional

The user who committed the Dataset Version.

committed_atdatetimeOptional

The date and time the Dataset Version was committed.

commit_messagestringOptional

Message describing the changes made. If provided, a committed version of the Dataset is created. Otherwise, an uncommitted version is created.

datapointslist of objectsOptional

The list of Datapoints in this Dataset version. Only provided if explicitly requested.

attributesmap from strings to anyOptional

Additional fields to describe the Dataset. Helpful to separate Dataset versions from each other with details on how they were created or used.

Errors

Get Dataset
Built with