February 2, 2024

Control logging level

We’ve added a save flag to all of our endpoints that generate logs on Humanloop so that you can control whether the request and response payloads that may contain sensitive information are persisted on our servers or not.

If save is set to false then no inputs, messages our outputs of any kind (including the raw provider request and responses) are stored on our servers. This can be helpful for sensitive use cases where you can’t for example risk PII leaving your system.

Details of the model configuration and any metadata you send are still stored. Therefore you can still benefit from certain types of evaluators such as human feedback, latency and cost, as well as still track important metadata over time that may not contain sensitive information.

This includes all our chat and completion endpoint variations, as well as our explicit log endpoint.

1
from humanloop import Humanloop
2
3
# You need to initialize the Humanloop SDK with your API Keys
4
humanloop = Humanloop(api_key="<YOUR Humanloop API KEY>")
5
6
# humanloop.complete_deployed(...) will call the active model config on your project.
7
# You can optionally set the save flag to False
8
complete_response = humanloop.complete_deployed(
9
  	save=False,
10
    project="<YOUR UNIQUE PROJECT NAME>",
11
    inputs={"question": "I have inquiry about by life insurance policy. Can you help?"},
12
)
13
14
# You can still retrieve the data_id and output as normal
15
data_id = complete_response.data[0].id
16
output = complete_response.data[0].output
17
18
# And log end user feedback that will still be stored
19
humanloop.feedback(data_id=data_id, type="rating", value="good")

Logging provider request

We’re now capturing the raw provider request body alongside the existing provider response for all logs generated from our deployed endpoints.

This provides more transparency into how we map our provider agnostic requests to specific providers. It can also effective for helping to troubleshoot the cases where we return well handled provider errors from our API.