Generate chat responses

The Humanloop Python SDK allows you to easily replace your openai.ChatCompletions.create() calls with a humanloop.chat() call that, in addition to calling OpenAI to get a response, automatically logs the data to your Humanloop project.

Prerequisites

• You already have a Prompt — if not, please follow our Prompt creation guide first.

Activate a model

1. Log in to Humanloop and navigate to the Models tab of your project.
2. Ensure that the default environment is in green at the top of the dashboard. The default environment is mapped to your active deployment. If there is no active deployment set, then use the dropdown button for the default environment and select the Change deployment option to select one of your existing model configs to use to generate. You also need to confirm the model you config you have deployed is a Chat model. This can be confirmed by clicking on the config in the table and viewing the Endpoint, making sure it says Chat.

Use the SDK to call your model

Now you can use the SDK to generate completions and log the results to your project:

1
# humanloop.chat_deployed(...) will call the active model config on your project.
2
# The inputs must match the input of the chat template in your project.
3
chat_response = humanloop.chat_deployed(
4
    project_id="YOUR_PROJECT_ID_HERE",
5
  	 # inputs required by your chat_template - for example your templated system message.
6
    inputs={"persona": "paul graham from YC"},
7
  	messages=[
8
    	  {"role": "user", "content": "How should I think about competition for my startup?"}
9
    ]
10
)
11
12
# A single call to chat may return multiple outputs.
13
data_id = chat_response.data[0].id
14
output = chat_response.data[0].output
15
print(output)
16
17
# You can also access the raw response from OpenAI.
18
print(chat_response.provider_responses)

Navigate to your project’s Logs tab in the browser to see the recorded inputs, messages and responses of your chat.

🎉 Now that you have chat messages flowing through your project you can start to log your end user feedback to evaluate and improve your models.