Create a Dataset from existing Logs

--- subtitle: >- In this guide, we will walk through creating a Dataset on Humanloop from your existing Logs. description: >- Learn how to create Datasets in Humanloop to define fixed examples for your projects, and build up a collection of input-output pairs for evaluation and fine-tuning. image: type: url value: 'https://humanloop.com/assets/docs/social-image.png' --- [Datasets](../../explanation/datasets) are a collection of input-output pairs that can be used to evaluate your Prompts, Tools or even Evaluators. This guide will show you how to create Datasets in Humanloop from your Logs. ### Prerequisites You should have an existing [Prompt](../../explanation/prompts) on Humanloop and already generated some [Logs](../../explanation/logs). Follow our guide on [creating a Prompt](../prompts/create-prompt). ## Steps To create a Dataset from existing Logs: <Steps> ### Navigate to the **Logs** of your Prompt Our Prompt in this example is a Support Agent that answers user queries about Humanloop's product and docs: <img src="file:8c92f42f-045b-4643-926b-85fc7417c5ff" alt="Navigate to the Logs table of your Prompt." /> ### Select a subset of the Logs to add Filter logs on a criteria of interest, such as the version of the Prompt used, then multi-select Logs. In the menu in the top right of the page, select **Add to Dataset**. <img src="file:82c2dfa0-47cc-41d3-bdc6-2536956691c3" alt="Filter and select logs of interest." /> ### Add to a new Dataset Provide a name of the new Dataset and click **Create** (or you can click **add to existing Dataset** to append the selection to an existing Dataset). Then provide a suitable commit message describing the datapoints you've added. <img src="file:0a20da32-256b-44c0-8901-549f8321be28" alt="Create a new dataset from logs." /> You will then see the new Dataset appear at the same level in the filesystem as your Prompt. </Steps> ## Next steps 🎉 Now that you have Datasets defined in Humanloop, you can leverage our [Evaluations](./overview) feature to systematically measure and improve the performance of your AI applications. See our guides on [setting up Evaluators](./llm-as-a-judge) and [Running an Evaluation](./run-evaluation) to get started. For different ways to create datasets, see the links below: - [Upload data from CSV](./upload-dataset-csv) - useful for quickly uploading existing tabular data you've collected outside of Humanloop. - [Upload via API](./create-dataset-api) - useful for uploading more complex Datasets that may have nested JSON structures, which are difficult to represent in tabular .CSV format, and for integrating with your existing data pipelines.