Logging through API

Instrument your AI project with any programming language using the API.

Our SDK offers high-level utilities for integrating Humanloop in your project. If you use a language without SDK support, or prefer more control, you can use the API directly to the same effect.

This guide revisits our logging quickstart tutorial: we’ll use API actions instead of the SDK decorators, showing you how Humanloop instrumentation works step by step.

By the end, we’ll have a chat agent project split into versioned components and with Humanloop logging integrated.

Prerequisites

Create the chat agent

We start with a simple chat agent that answers math and science questions.

Create an agent.py file and add the following:

agent.py
1import os
2import json
3import datetime
4
5from humanloop import Humanloop
6from openai import OpenAI
7
8openai = OpenAI(api_key="YOUR_OPENAI_KEY")
9humanloop = Humanloop(api_key="YOUR_HUMANLOOP_KEY")
10
11
12def calculator(operation: str, num1: int, num2: int) -> str:
13    """Do arithmetic operations on two numbers."""
14    if operation == "add":
15        return num1 + num2
16    elif operation == "subtract":
17        return num1 - num2
18    elif operation == "multiply":
19        return num1 * num2
20    elif operation == "divide":
21        return num1 / num2
22    else:
23        return "Invalid operation"
24
25
26TOOL_JSON_SCHEMA = {
27    "name": "calculator",
28    "description": "Do arithmetic operations on two numbers.",
29    "parameters": {
30        "type": "object",
31        "required": ["operation", "num1", "num2"],
32        "properties": {
33            "operation": {"type": "string"},
34            "num1": {"type": "integer"},
35            "num2": {"type": "integer"},
36        },
37        "additionalProperties": False,
38    },
39}
40
41
42def call_model(messages: list[str]) -> str:
43    output = openai.chat.completions.create(
44        messages=messages,
45        model="gpt-4o",
46        tools=[
47            {
48                "type": "function",
49                "function": TOOL_JSON_SCHEMA,
50            }
51        ],
52        temperature=0.7,
53    )
54
55    # Check if model asked for a tool call
56    if output.choices[0].message.tool_calls:
57        for tool_call in output.choices[0].message.tool_calls:
58            arguments = json.loads(tool_call.function.arguments)
59            if tool_call.function.name == "calculator":
60                result = calculator(**arguments)
61                return f"[TOOL CALL] {result}"
62
63    # Otherwise, return the LLM response
64    return output.choices[0].message.content
65
66
67def conversation():
68    messages = [
69        {
70            "role": "system",
71            "content": "You are a a groovy 80s surfer dude "
72            "helping with math and science.",
73        },
74    ]
75    while True:
76        user_input = input("You: ")
77        if user_input == "exit":
78            break
79        messages.append({"role": "user", "content": user_input})
80        response = call_model(messages=messages)
81        messages.append({"role": "assistant", "content": response})
82        print(f"Agent: {response}")
83
84
85if __name__ == "__main__":
86    conversation()

Log to Humanloop

Use the SDK to create the versionable components of the agent. Logging against a File on a given path will create the File if it does not exist. Changing the File definition will create a new version of the File.

1

Initialize the trace

Modify call_model to accept a trace_id argument. It will be used to associate Logs to the logging trace.

The trace of the conversation will be contained by a Flow. Initialize the trace at the start of the conversation.

agent.py
1def call_model(trace_id: str, messages: list[str]) -> str:
2    ...
3
4def conversation():
5    trace_id = humanloop.flows.log(
6        path="Logging Quickstart/QA Agent",
7        flow={
8            "attributes": {},
9        },
10    ).id
11    messages = [
12        {
13            "role": "system",
14            "content": "You are a a groovy 80s surfer dude "
15            "helping with math and science.",
16        },
17    ]
18    while True:
19        user_input = input("You: ")
20        if user_input == "exit":
21            break
22        messages.append({"role": "user", "content": user_input})
23        response = call_model(trace_id=trace_id, messages=messages)
24        messages.append({"role": "assistant", "content": response})
25        print(f"Agent: {response}")
2

Add logging

We add log statements that will create the Logs contained in the trace.

agent.py
1def call_model(trace_id: str, messages: list[str]) -> str:
2    prompt_start_time = datetime.datetime.now()
3    output = openai.chat.completions.create(
4        messages=messages,
5        model="gpt-4o",
6        tools=[{
7            "type": "function",
8            "function": TOOL_JSON_SCHEMA,
9        }],
10        temperature=0.7,
11    )
12    prompt_log_id = humanloop.prompts.log(
13        path="Logging Quickstart/QA Prompt",
14        prompt={
15            "model": "gpt-4o",
16            "tools": [TOOL_JSON_SCHEMA],
17            "temperature": 0.7,
18        },
19        output=output.choices[0].message.content,
20        trace_parent_id=trace_id,
21        start_time=prompt_start_time,
22        end_time=datetime.datetime.now(),
23    ).id
24
25    # Check if model asked for a tool call
26    if output.choices[0].message.tool_calls:
27        for tool_call in output.choices[0].message.tool_calls:
28            arguments = json.loads(tool_call.function.arguments)
29            if tool_call.function.name == "calculator":
30                tool_start_time = datetime.datetime.now()
31                result = calculator(**arguments)
32                humanloop.tools.log(
33                    path="Logging Quickstart/Calculator",
34                    tool={
35                        "name": "calculator",
36                        "description": "Do arithmetic operations on two numbers.",
37                        "function": TOOL_JSON_SCHEMA,
38                    },
39                    inputs=arguments,
40                    output=result,
41                    trace_parent_id=prompt_log_id,
42                    start_time=tool_start_time,
43                    end_time=datetime.datetime.now(),
44                )
45                return f"[TOOL CALL] {result}"
46
47    # Otherwise, return the LLM response
48    return output.choices[0].message.content
3

Complete the trace

When the conversation is finished, we mark the trace as complete, signalling no more logs will be added.

agent.py
1def conversation():
2    trace_id = humanloop.flows.log(
3        path="Logging Quickstart/QA Agent",
4        flow={
5            "attributes": {},
6        },
7    ).id
8    messages = [
9        {
10            "role": "system",
11            "content": "You are a a groovy 80s surfer dude "
12            "helping with math and science.",
13        },
14    ]
15    while True:
16        user_input = input("You: ")
17        if user_input == "exit":
18            break
19        messages.append({"role": "user", "content": user_input})
20        response = call_model(trace_id=trace_id, messages=messages)
21        messages.append({"role": "assistant", "content": response})
22        print(f"Agent: {response}")
23
24    humanloop.flows.update_log(
25        log_id=trace_id,
26        output="",
27        status="complete",
28    )

Run the code

Have a conversation with the agent. When you’re done, type exit to close the program.

$python agent.py
>You: Hi dude!
>Agent: Tubular! I am here to help with math and science, what is groovin?
>You: How does flying work?
>Agent: ...
>You: What is 5678 * 456?
>Agent: [TOOL CALL] 2587968
>You: exit

Check your workspace

Navigate to your workspace to see the logged conversation.

Inside the Logging Quickstart directory on the left, click the QA Agent Flow. Select the Logs tab from the top of the page and click the Log inside the table.

You will see the conversation’s trace, containing Logs corresponding to the Tool and the Prompt.

Change the agent and rerun

Modify the call_model function to use a different model and temperature.

agent.py
1def call_model(trace_id: str, messages: list[str]) -> str:
2    prompt_start_time = datetime.datetime.now()
3    output = openai.chat.completions.create(
4        messages=messages,
5        model="gpt-4o-mini",
6        tools=[{
7            "type": "function",
8            **TOOL_JSON_SCHEMA,
9        }],
10        temperature=0.2,
11    )
12    prompt_log_id = humanloop.prompts.log(
13        path="Logging Quickstart/QA Prompt",
14        prompt={
15            "model": "gpt-4o-mini",
16            "tools": [TOOL_JSON_SCHEMA],
17            "temperature": 0.2,
18        }
19        output=output.choices[0].message.content,
20        trace_parent_id=trace_id,
21        start_time=prompt_start_time,
22        end_time=datetime.datetime.now(),
23    ).id
24
25    ...

Run the agent again, then head back to your workspace.

Click the QA Prompt Prompt, select the Dashboard tab from the top of the page and look at Uncommitted Versions.

By changing the hyperparameters of the OpenAI call, you have tagged a new version of the Prompt.

Next steps

Logging is the first step to observing your AI product. Follow up with these guides on monitoring and evals: