Set up CI/CD Evaluations

In this guide, we will walk through setting up CI/CD integration for Humanloop evaluations using GitHub Actions.

Paid Feature

This feature is not available for the Free tier. Please contact us if you wish to learn more about our Enterprise plan

Setting up CI/CD Integration with GitHub Actions

Integrating Humanloop evaluations into your CI/CD pipeline allows you to automatically test your AI applications as part of your development workflow. This guide will walk you through setting up this integration using GitHub Actions.

Prerequisites

  • A GitHub repository for your project
  • A Humanloop account with access to Evaluations
  • A Prompt and Dataset set up in Humanloop
  • An Evaluator configured in Humanloop

Steps to Set Up CI/CD Integration

1

Create a GitHub Actions Workflow

In your GitHub repository, create a new file .github/workflows/humanloop-eval.yml with the following content:

Under Development

This content is currently under development. Please refer to our V4 documentation for the current docs.