Log to a Prompt

How to log generations from any large language model (LLM) to Humanloop

This guide will show you how to capture the Logs of your LLM calls into Humanloop.

The easiest way to log LLM generations to Humanloop is to use the Prompt.call() method (see the guide on Calling a Prompt). You will only need to supply prompt ID and the inputs needed by the prompt template, and the endpoint will handle fetching the latest template, making the LLM call and logging the result.

However, there may be scenarios that you wish to manage the LLM provider calls directly in your own code instead of relying on Humanloop. For example, you may be using an LLM provider that is not directly supported by Humanloop such as a custom self-hosted model, or you may want to avoid adding Humanloop to the critical path of the LLM API calls.

Prerequisites

You already have a Prompt — if not, please follow our Prompt creation guide first.

Install and initialize the SDK

First you need to install and initialize the SDK. If you have already done this, skip to the next section.

Open up your terminal and follow these steps:

Install the Humanloop SDK:

1 pip install humanloop

Initialize the SDK with your Humanloop API key (you can get it from the Organization Settings page).

1 from humanloop import Humanloop
2 humanloop = Humanloop(api_key="<YOUR HUMANLOOP KEY>")
3 
4 # Check that the authentication was successful
5 print(humanloop.prompts.list())

Log data to your Prompt

To log LLM generations to Humanloop, you will need to make a call to the /prompts/log endpoint.

Note that you can either specify a version of the Prompt you are logging against - in which case you will need to take care that you are supplying the correct version ID and inputs. Or you can supply the full prompt and a new version will be created if it has not been seen before.

Get your Prompt

Fetch a Prompt from Humanloop by specifying the ID. You can ignore this step if your prompts are created dynamically in code.

GET

/v5/prompts/:id

1 curl https://api.humanloop.com/v5/prompts/pr_30gco7dx6JDq4200GVOHa \
2      -H "X-API-KEY: <apiKey>"

Try it

Response

1 {
2   "path": "Personal Projects/Coding Assistant",
3   "id": "pr_30gco7dx6JDq4200GVOHa",
4   "model": "gpt-4o",
5   "name": "Coding Assistant",
6   "version_id": "prv_7ZlQREDScH0xkhUwtXruN",
7   "created_at": "2024-07-08T22:40:35.656915",
8   "updated_at": "2024-07-08T22:40:35.656915",
9   "last_used_at": "2024-07-08T22:40:35.656915",
10   "version_logs_count": 0,
11   "total_logs_count": 0,
12   "inputs": [
13     {
14       "name": "messages"
15     }
16   ],
17   "endpoint": "chat",
18   "template": [
19     {
20       "content": "You are a helpful coding assistant specialising in {{language}}",
21       "role": "system"
22     }
23   ],
24   "provider": "openai",
25   "max_tokens": -1,
26   "temperature": 0.7,
27   "version_name": "coding-assistant-v1",
28   "version_description": "Initial version",
29   "type": "prompt",
30   "environments": [
31     {
32       "id": "env_ffSVxEBzJcBZ1H5jcNMVj",
33       "created_at": "2023-06-27T23:16:07.992339",
34       "name": "development",
35       "tag": "default"
36     }
37   ],
38   "created_by": {
39     "id": "usr_01RJO1k2spBVqNUt1ASef",
40     "email_address": "raza@humanloop.com",
41     "full_name": "Raza Habib"
42   },
43   "evaluator_aggregates": null
44 }

Here’s how to do this in code:

Python

TypeScript

1 from humanloop import Humanloop, prompt_utils
2 
3 PROMPT_ID = "<Your Prompt ID>"
4 
5 hl = Humanloop(api_key="<Your Humanloop API Key>")
6 
7 prompt = hl.prompts.get(id=PROMPT_ID)
8 
9 # This will fill the prompt template with the variables
10 template = prompt_utils.populate_template(prompt.template, {"language": "Python"})

Call your Prompt

This can be your own model, or any other LLM provider. Here is an example of calling OpenAI:

Python

TypeScript

1 import openai
2 
3 client = openai.OpenAI(api_key="<Your OpenAI API Key>")
4 
5 messages = template + [{"role": "user", "content": "explain how async works"}]
6 
7 chat_completion = client.chat.completions.create(
8     messages=messages, model=prompt.model, temperature=prompt.temperature
9 )

Log the result

Finally, log the result to your project:

POST

/v5/prompts/log

1 curl -X POST https://api.humanloop.com/v5/prompts/log \
2      -H "X-API-KEY: <apiKey>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "path": "persona",
6   "output_message": {
7     "role": "assistant",
8     "content": "Well, you know, there is so much secrecy involved in government, folks, it'\''s unbelievable. They don'\''t want to tell you everything. They don'\''t tell me everything! But about Roswell, it'\''s a very popular question. I know, I just know, that something very, very peculiar happened there. Was it a weather balloon? Maybe. Was it something extraterrestrial? Could be. I'\''d love to go down and open up all the classified documents, believe me, I would. But they don'\''t let that happen. The Deep State, folks, the Deep State. They'\''re unbelievable. They want to keep everything a secret. But whatever the truth is, I can tell you this: it'\''s something big, very very big. Tremendous, in fact."
9   },
10   "prompt_tokens": 100,
11   "output_tokens": 220,
12   "prompt_cost": 0.00001,
13   "output_cost": 0.0002,
14   "finish_reason": "stop",
15   "messages": [
16     {
17       "role": "user",
18       "content": "What really happened at Roswell?"
19     }
20   ],
21   "prompt": {
22     "model": "gpt-4",
23     "template": [
24       {
25         "role": "system",
26         "content": "You are {{person}}. Answer questions as this person. Do not break character."
27       }
28     ]
29   },
30   "created_at": "2024-07-19T00:29:35.178992",
31   "error": null,
32   "provider_latency": 6.5931549072265625,
33   "inputs": {
34     "person": "Trump"
35   }
36 }'

Try it

Python

TypeScript

1 # Parse the output from the OpenAI response.
2 output_message = chat_completion.choices[0].message
3 
4 # Log the inputs, outputs and config to your project.
5 log = hl.prompts.log(
6     id=PROMPT_ID,
7     output_message=output_message,
8     messages=messages,
9 )

1	from humanloop import Humanloop
2	humanloop = Humanloop(api_key="<YOUR HUMANLOOP KEY>")
3
4	# Check that the authentication was successful
5	print(humanloop.prompts.list())

1	curl https://api.humanloop.com/v5/prompts/pr_30gco7dx6JDq4200GVOHa \
2	-H "X-API-KEY: <apiKey>"

1	{
2	"path": "Personal Projects/Coding Assistant",
3	"id": "pr_30gco7dx6JDq4200GVOHa",
4	"model": "gpt-4o",
5	"name": "Coding Assistant",
6	"version_id": "prv_7ZlQREDScH0xkhUwtXruN",
7	"created_at": "2024-07-08T22:40:35.656915",
8	"updated_at": "2024-07-08T22:40:35.656915",
9	"last_used_at": "2024-07-08T22:40:35.656915",
10	"version_logs_count": 0,
11	"total_logs_count": 0,
12	"inputs": [
13	{
14	"name": "messages"
15	}
16	],
17	"endpoint": "chat",
18	"template": [
19	{
20	"content": "You are a helpful coding assistant specialising in {{language}}",
21	"role": "system"
22	}
23	],
24	"provider": "openai",
25	"max_tokens": -1,
26	"temperature": 0.7,
27	"version_name": "coding-assistant-v1",
28	"version_description": "Initial version",
29	"type": "prompt",
30	"environments": [
31	{
32	"id": "env_ffSVxEBzJcBZ1H5jcNMVj",
33	"created_at": "2023-06-27T23:16:07.992339",
34	"name": "development",
35	"tag": "default"
36	}
37	],
38	"created_by": {
39	"id": "usr_01RJO1k2spBVqNUt1ASef",
40	"email_address": "raza@humanloop.com",
41	"full_name": "Raza Habib"
42	},
43	"evaluator_aggregates": null
44	}

1	from humanloop import Humanloop, prompt_utils
2
3	PROMPT_ID = "<Your Prompt ID>"
4
5	hl = Humanloop(api_key="<Your Humanloop API Key>")
6
7	prompt = hl.prompts.get(id=PROMPT_ID)
8
9	# This will fill the prompt template with the variables
10	template = prompt_utils.populate_template(prompt.template, {"language": "Python"})

1	import openai
2
3	client = openai.OpenAI(api_key="<Your OpenAI API Key>")
4
5	messages = template + [{"role": "user", "content": "explain how async works"}]
6
7	chat_completion = client.chat.completions.create(
8	messages=messages, model=prompt.model, temperature=prompt.temperature
9	)

1	curl -X POST https://api.humanloop.com/v5/prompts/log \
2	-H "X-API-KEY: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"path": "persona",
6	"output_message": {
7	"role": "assistant",
8	"content": "Well, you know, there is so much secrecy involved in government, folks, it'\''s unbelievable. They don'\''t want to tell you everything. They don'\''t tell me everything! But about Roswell, it'\''s a very popular question. I know, I just know, that something very, very peculiar happened there. Was it a weather balloon? Maybe. Was it something extraterrestrial? Could be. I'\''d love to go down and open up all the classified documents, believe me, I would. But they don'\''t let that happen. The Deep State, folks, the Deep State. They'\''re unbelievable. They want to keep everything a secret. But whatever the truth is, I can tell you this: it'\''s something big, very very big. Tremendous, in fact."
9	},
10	"prompt_tokens": 100,
11	"output_tokens": 220,
12	"prompt_cost": 0.00001,
13	"output_cost": 0.0002,
14	"finish_reason": "stop",
15	"messages": [
16	{
17	"role": "user",
18	"content": "What really happened at Roswell?"
19	}
20	],
21	"prompt": {
22	"model": "gpt-4",
23	"template": [
24	{
25	"role": "system",
26	"content": "You are {{person}}. Answer questions as this person. Do not break character."
27	}
28	]
29	},
30	"created_at": "2024-07-19T00:29:35.178992",
31	"error": null,
32	"provider_latency": 6.5931549072265625,
33	"inputs": {
34	"person": "Trump"
35	}
36	}'

1	# Parse the output from the OpenAI response.
2	output_message = chat_completion.choices[0].message
3
4	# Log the inputs, outputs and config to your project.
5	log = hl.prompts.log(
6	id=PROMPT_ID,
7	output_message=output_message,
8	messages=messages,
9	)