Create a Dataset via the API

Datasets are a collection of input-output pairs that can be used to evaluate your Prompts, Tools or even Evaluators.

Prerequisites

If you are using the SDK, the only prerequisite is to have the SDK installed and configured. If you are using the API directly, you will need to have an API key.

Install and initialize the SDK

Steps

Using the API is a great way to integrate Humanloop with your existing data pipeline or just to once-off upload a more complex Dataset that is hard to represent in a CSV file, such as one that contains an array of messages and JSON targets.

Post data to the Datasets API

We first define some sample data that contains user messages and desired responses from our Support Agent Prompt and call the POST /datasets endpoint to upload it as follows:

Inspect the uploaded Dataset

After running this code, in your Humanloop workspace you will now see a Dataset called Support Query Ground Truth (or whatever value was in path) with your sample data.

Next steps

🎉 Now that you have Datasets defined in Humanloop, you can leverage our Evaluations feature to systematically measure and improve the performance of your AI applications. See our guides on setting up Evaluators and Running an Evaluation to get started.

For different ways to create datasets, see the links below:

Create a Dataset from existing Logs - useful for curating Datasets based on how your AI application has been behaving in the wild.
Upload data from CSV - useful for quickly uploading existing tabular data you’ve collected outside of Humanloop.