Capture user feedback

Prerequisites

• You already have a Prompt — if not, please follow our Prompt creation guide first.
• You have created a Human Evaluator. For this guide, we will use the “rating” example Evaluator automatically created for your organization.

Configure feedback

To collect user feedback, connect a Human Evaluator to your Prompt. The Evaluator specifies the type of the feedback you want to collect. See our guide on creating Human Evaluators for more information.

1

Open the Prompt’s monitoring dialog

Go to your Prompt’s dashboard. Click Monitoring in the top right to open the monitoring dialog.

2

Connect your Evaluator

Click Connect Evaluators and select the Human Evaluator you created.

You should now see the selected Human Evaluator attached to the Prompt in the Monitoring dialog.

Log feedback

With the Human Evaluator attached to your Prompt, you can record feedback against the Prompt’s Logs.

1
log = humanloop.prompts.call(
2
    version_id="prv_qNeXZp9P6T7kdnMIBHIOV",
3
    path="persona",
4
    messages=[{"role": "user", "content": "What really happened at Roswell?"}],
5
    inputs={"person": "Trump"},
6
)
7
log_id = log.id

1
feedback = humanloop.evaluators.log(
2
    # Pass the `log_id` from the previous step to indicate the Log to record feedback against
3
    parent_id=log_id,
4
    # Here, we're recording feedback against a "Tweet Issues" Human Evaluator,
5
    # which is of type `multi_select` and has multiple options to choose from.
6
    path="Feedback Demo/Tweet Issues",
7
    judgment=["Inappropriate", "Too many emojis"],
8
)

View feedback

You can view the applied feedback in two main ways: through the Logs that the feedback was applied to, and through the Evaluator itself.

Feedback applied to Logs

The feedback recorded for each Log can be viewed in the Logs table of your Prompt.

Your internal users can also apply feedback to the Logs directly through the Humanloop app.

Feedback for an Evaluator

You can view all feedback recorded for a specific Human Evaluator in the Logs tab of the Evaluator. This will display all feedback recorded for the Evaluator across all other Files.

Next steps

• Create and customize your own Human Evaluators to capture the feedback you need.
• Human Evaluators can also be used in Evaluations, allowing you to collect judgments from your subject-matter experts.