> This page is for version v5.0 (default).
> For other versions, use one of these documentation indexes:
> - v5.0 (default): https://humanloop.com/docs/v5/llms.txt
> - v4.0: https://humanloop.com/docs/v4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://humanloop.com/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://humanloop.com/docs/_mcp/server.

> Run an Evaluation on Humanloop with your own

This guide demonstrates how to run an Evaluation on Humanloop using your own logs.
This is useful if you have existing logs in an external system and want to evaluate them on Humanloop
with minimal setup.

In this guide, we will use the example of a JSON file containing chat messages between users
and customer support agents. This guide will bring you through uploading these logs to Humanloop
and creating an Evaluation with them.

## Prerequisites

The code in this guide uses the Python SDK. To follow along, you will need to have the SDK installed and configured.
While the code snippets are in Python, the same steps can be performed using the TypeScript SDK or via the API directly.
If you are using the API directly, you will need to have an API key.

#### Install and initialize the SDK

First you need to install and initialize the SDK. If you have already done this, skip to the next section.

Open up your terminal and follow these steps:

1. Install the Humanloop SDK:

```python
pip install humanloop
```

```typescript
npm install humanloop
```

2. Initialize the SDK with your Humanloop API key (you can get it from the [Organization Settings page](https://app.humanloop.com/account/api-keys)).

```python
from humanloop import Humanloop
humanloop = Humanloop(api_key="<YOUR HUMANLOOP KEY>")

# Check that the authentication was successful
print(humanloop.prompts.list())
```

```typescript
import { HumanloopClient, Humanloop } from "humanloop";

const humanloop = new HumanloopClient({ apiKey: "YOUR_API_KEY" });

// Check that the authentication was successful
console.log(await humanloop.prompts.list());
```

The example JSON data in this guide can be found in the [Humanloop Cookbook](https://github.com/humanloop/humanloop-cookbook/tree/evaluating-external-logs/assets).
To continue with the code in this guide, download `conversations-a.json` and `conversations-b.json` from the `assets` folder.

## Evaluate your external logs

We'll start by loading data from `conversations-a.json`, which represents logs recorded by an external system.

```python
import json

with open("conversations-a.json") as f:
    data = json.load(f)
```

In this example, `data` is a list of chat messages between users and a support agent.

### Upload logs to Humanloop

> **Note**
>
> These steps are suitable if you do not already have an Evaluation on Humanloop.
> The [Upload new logs step](#upload-new-logs) demonstrates a simpler process
> if you already have an Evaluation you want to add a new set of logs to.

Upload the logs with the `log(...)` method.
This will automatically create a [Flow](../../explanation/flows) on Humanloop.

We additionally pass in some `attributes` identifying
the configuration of the system that generated these logs.
`attributes` accepts arbitrary values, and is used for versioning the Flow.
Here, it allows us to associate this set of logs with a specific version of the support agent.

```python
log_ids = []
for messages in data:
    log = humanloop.flows.log(
        path="External logs demo/Travel planner",
        flow={"attributes": {"agent-version": "1.0.0"}},  # Optionally add attributes to identify this version of the support agent.
        messages=messages,
    )
    log_ids.append(log.id)
```

This will have created a new Flow on Humanloop named **Travel planner**.
To confirm this logging has succeeded, navigate to the **Logs** tab of the Flow
and view the uploaded logs. Each Log should correspond to a conversation and contain
a list of messages.

![Flow Logs](/docs/_fern-img/6ce76d2d7dd9016781ced002b38129edc62cba3ea29500762dd09dc7a1192e63.webp)

We will also use the created Flow version when creating our Run.

```python
version_id = log.version_id
```

### Create an Evaluation Run

Next, create an Evaluation on Humanloop. Within the Evaluation, create a Run which will contain the Logs.

Here, we'll use the example "Helpfulness" LLM-as-a-judge Evaluator. This will automatically rate
the helpfulness of the support agent across our logs.

```python
evaluation = humanloop.evaluations.create(
    name="Past records",
    # NB: you can use `path`or `id` for references on Humanloop
    file={"path": "External logs demo/Travel planner"},
    evaluators=[
        # Replace with your Evaluators
        {"path": "Example Evaluators/AI/Helpfulness"},
    ],
)
run = humanloop.evaluations.create_run(
    id=evaluation.id,
    version={'version_id': version_id},  # Associate this Run to the Flow version created above.
)
```

### Assign Logs to the Run

Finally, add the Flow Logs to the Run.

```python
humanloop.evaluations.add_logs_to_run(
    id=evaluation.id,
    run_id=run.id,
    log_ids=log_ids,
)
```

### Review the Evaluation

You have now created an Evaluation on Humanloop and added Logs to it.

![Evaluation on Humanloop](/docs/_fern-img/a0f6d412837445bd5853735ad71f9b94c6e345dfc7b79eb3434b890af76b6bf2.webp)

Go to the Humanloop UI to view the Evaluation. Within the Evaluation, go to **Logs** tab.
Here, you can view your uploaded logs as well as the Evaluator judgments.

![Logs tab of Evaluation](/docs/_fern-img/d9478dc1893f4b1466d5ff39d96b2b72e3af74590ff4eea0a5481c3f567fb20d.webp)

The following steps will guide you through adding a different set of logs to a new Run
for comparison.

### Upload new logs

> **Info**
>
> If you already have an Evaluation that you want to add a new set of logs to,
> you can start from here.
>
> To start from this point, retrieve the ID of the Evaluation you want to add logs to.
>
> Go to the Evaluation you want to add logs to on the Humanloop UI and copy the ID
> from the URL. This is the segment of the URL after `evaluations/`, e.g. `evr_...`.

Now that we have an Evaluation on Humanloop, we can add a separate set of logs to it
and compare the performance across this set of logs to the previous set.

While we can achieve this by repeating the above steps, we can add logs to a Run
in a more direct and simpler way now that we have an existing Evaluation.

For this example, we'll continue with the Evaluation created in the previous section,
and add a new Run with the data from `conversations-b.json`. These represent a set of logs
from a prototype version of the support agent.

```python
with open("conversations-b.json") as f:
    data = json.load(f)
```

#### Create a new Run

Create a new Run within the Evaluation that will contain this set of logs.

```python
run = humanloop.evaluations.create_run(
    id=evaluation.id,
)
```

#### Log to the Run

Pass the `run_id` argument in your `log(...)` call to associate the Log with the Run.

```python
# Add the new data to the Run
for messages in data:
    log = humanloop.flows.log(
        path="External logs demo/Travel planner",
        flow={"attributes": {"agent-version": "2.0.0"}},
        messages=messages,
        # Pass `run_id` to associate the Log with the Run.
        run_id=run.id,
    )
```

### Compare the results

View the Evaluation on Humanloop. It will now contain two Runs.

In the **Stats** tab of the Evaluation, you can now compare the performance of the two sets of logs.

In our case, our second set of logs (on the right) can be seen to be less helpful.

![Evaluation with two Runs on Humanloop](/docs/_fern-img/43b9dcccabdb8cd9c131330a6a0155c3b59aca61b0aa3cd74542968a7f3dc60a.webp)

![Stats tab showing box plots for the two Runs](/docs/_fern-img/8aba75b424844258ae62448e24c04d0eccbc615b5e4f9f17aea6f06b85e36299.webp)

## Next steps

The above examples demonstrate how you can quickly populate an Evaluation Run with your logs.

* You can extend this Evaluation with custom Evaluators, such as using [Code Evaluators](./code-based-evaluator)
  to calculate metrics, or using [Human Evaluators](/docs/guides/evals/human-evaluators)
  to set up your Logs to be reviewed by your subject-matter experts.
* Now that you've set up an Evaluation, explore the other [File](../../explanation/files) types on Humanloop to see how they can better reflect
  your production systems, and how you can use Humanloop to version-control them. Here, we've used a [Flow](../../explanation/flows) to represent a black-box system.