Generate chat responses

A walkthrough of how to generate chat completions from a large language model with the prompt managed in Humanloop.

The Humanloop Python SDK allows you to easily replace your openai.ChatCompletions.create() calls with a humanloop.chat() call that, in addition to calling OpenAI to get a response, automatically logs the data to your Humanloop project.

Prerequisites

You already have a Prompt — if not, please follow our Prompt creation guide first.

This guide assumes you’re using an OpenAI model. If you want to use other providers or your own model please also look at our guide to using your own model.

Install and initialize the SDK

First you need to install and initialize the SDK. If you have already done this, skip to the next section.

Open up your terminal and follow these steps:

Install the Humanloop SDK:

1 pip install humanloop

Initialize the SDK with your Humanloop API key (you can get it from the Organization Settings page).

1 from humanloop import Humanloop
2 humanloop = Humanloop(api_key="<YOUR HUMANLOOP KEY>")
3 
4 # Check that the authentication was successful
5 print(humanloop.prompts.list())

Activate a model

Log in to Humanloop and navigate to the Models tab of your project.
Ensure that the default environment is in green at the top of the dashboard. The default environment is mapped to your active deployment. If there is no active deployment set, then use the dropdown button for the default environment and select the Change deployment option to select one of your existing model configs to use to generate. You also need to confirm the model you config you have deployed is a Chat model. This can be confirmed by clicking on the config in the table and viewing the Endpoint, making sure it says Chat.

Use the SDK to call your model

Now you can use the SDK to generate completions and log the results to your project:

1 # humanloop.chat_deployed(...) will call the active model config on your project.
2 # The inputs must match the input of the chat template in your project.
3 chat_response = humanloop.chat_deployed(
4     project_id="YOUR_PROJECT_ID_HERE",
5   	 # inputs required by your chat_template - for example your templated system message.
6     inputs={"persona": "paul graham from YC"},
7   	messages=[
8     	  {"role": "user", "content": "How should I think about competition for my startup?"}
9     ]
10 )
11 
12 # A single call to chat may return multiple outputs.
13 data_id = chat_response.data[0].id
14 output = chat_response.data[0].output
15 print(output)
16 
17 # You can also access the raw response from OpenAI.
18 print(chat_response.provider_responses)

Navigate to your project’s Logs tab in the browser to see the recorded inputs, messages and responses of your chat.

🎉 Now that you have chat messages flowing through your project you can start to log your end user feedback to evaluate and improve your models.

1	from humanloop import Humanloop
2	humanloop = Humanloop(api_key="<YOUR HUMANLOOP KEY>")
3
4	# Check that the authentication was successful
5	print(humanloop.prompts.list())

1	# humanloop.chat_deployed(...) will call the active model config on your project.
2	# The inputs must match the input of the chat template in your project.
3	chat_response = humanloop.chat_deployed(
4	project_id="YOUR_PROJECT_ID_HERE",
5	# inputs required by your chat_template - for example your templated system message.
6	inputs={"persona": "paul graham from YC"},
7	messages=[
8	{"role": "user", "content": "How should I think about competition for my startup?"}
9	]
10	)
11
12	# A single call to chat may return multiple outputs.
13	data_id = chat_response.data[0].id
14	output = chat_response.data[0].output
15	print(output)
16
17	# You can also access the raw response from OpenAI.
18	print(chat_response.provider_responses)