Set up Webhooks
In this guide, we will demonstrate how to set up webhooks via API for alerting on your monitoring evaluators.
Under Development
This content is currently under development. Please refer to our V4 documentation for the current docs.
Paid Feature
This feature is not available for the Free tier. Please contact us if you wish to learn more about our Enterprise plan
In this guide, we’ll walk you through the process of setting up webhooks using the Humanloop API to notify you in Slack when certain events occur with your monitoring evaluators.
Prerequisites
Before you begin, make sure you have:
- A Humanloop account with API access
- A Slack workspace where you have permissions to add webhooks
- A Humanloop project with at least one LLM model and monitoring evaluator set up
Setting up a webhook
To set up a webhook, you’ll use the hl.webhook.create()
method from the Humanloop Python SDK. Here’s a step-by-step guide:
Create a Slack incoming webhook
- Go to your Slack workspace and create a new Slack app (or use an existing one).
- Under “Add features and functionality”, choose “Incoming Webhooks” and activate them.
- Click “Add New Webhook to Workspace” and choose the channel where you want to receive notifications.
- Copy the webhook URL provided by Slack.
Import the Humanloop SDK and initialize the client
Replace "your-api-key"
with your actual Humanloop API key.
Verifying the webhook
After setting up the webhook and triggering an evaluation, you should see a message in your specified Slack channel. The message will contain details about the evaluation event, such as:
Managing webhooks
You can list, update, or delete webhooks using the following methods:
Replace "webhook-id"
with the ID of the webhook you want to manage.
Conclusion
You’ve now set up a webhook to receive notifications in Slack when your monitoring evaluators complete evaluations or detect drift. This will help you stay informed about the performance and behavior of your LLM models in real-time.