Prompts define how a large language model behaves.

A Prompt on Humanloop encapsulates the instructions and other configuration for how a large language model should perform a specific task. Each change in any of the following properties creates a new version of the Prompt:

  • the template such as Write a song about {{topic}}. For chat models, your template will contain an array of messages.
  • the model e.g. gpt-4o
  • all the parameters to the model such as temperature, max_tokens, top_p etc.
  • any tools available to the model

A Prompt is callable in that if you supply the necessary inputs, it will return a response from the model.

Inputs are defined in the template through the double-curly bracket syntax e.g. {{topic}} and the value of the variable will need to be supplied when you call the Prompt to create a generation.

This separation of concerns, keeping configuration separate from the query time data, is crucial for enabling you to experiment with different configurations and evaluate any changes. The Prompt stores the configuration and the query time data in Logs, which can then be used to create Datasets for evaluation purposes.

Note that we use a capitalized “Prompt” to refer to the entity in Humanloop, and a lowercase “prompt” to refer to the general concept of input to the model.

1---
2model: gpt-4
3temperature: 1.0
4max_tokens: -1
5provider: openai
6endpoint: chat
7---
8<system>
9 Write a song about {{topic}}
10</system>
An example Prompt, serialized as a Promptfile

Versioning

A Prompt file will have multiple versions as you try out different models, params or templates, but they should all be doing the same task, and in general should be swappable with one-another.

By versioning your Prompts, you can track how adjustments to the template or parameters influence the LLM’s responses. This is crucial for iterative development, as you can pinpoint which versions produce the most relevant or accurate outputs for your specific use case.

When to create a new Prompt

You should create a new Prompt for every different ‘task to be done’ with the LLM. For example each of these tasks are things that can be done by an LLM and should be a separate Prompt File: Writing Copilot, Personal Assistant, Summariser, etc.

We’ve seen people find it useful to also create a Prompt called ‘Playground’ where they can free form experiment without concern of breaking anything or making a mess of their other Prompts.

Using Prompts

Prompts are callable as an API. You supply and query-time data such as input values or user messages, and the model will respond with its text output.

POST
1curl -X POST https://api.humanloop.com/v5/prompts/call \
2 -H "X-API-KEY: <apiKey>" \
3 -H "Content-Type: application/json" \
4 -d '{
5 "stream": false,
6 "path": "persona",
7 "messages": [
8 {
9 "role": "user",
10 "content": "latest apple"
11 }
12 ],
13 "prompt": {
14 "model": "gpt-4",
15 "template": [
16 {
17 "role": "system",
18 "content": "You are stockbot. Return latest prices."
19 }
20 ],
21 "tools": [
22 {
23 "name": "get_stock_price",
24 "description": "Get current stock price",
25 "parameters": {
26 "type": "object",
27 "properties": {
28 "ticker_symbol": {
29 "type": "string",
30 "name": "Ticker Symbol",
31 "description": "Ticker symbol of the stock"
32 }
33 },
34 "required": []
35 }
36 }
37 ]
38 }
39}'

You can also use Prompts without proxying through Humanloop to the model provider and instead call the model yourself and explicitly log the results to your Prompt.

POST
1curl -X POST https://api.humanloop.com/v5/prompts/log \
2 -H "X-API-KEY: <apiKey>" \
3 -H "Content-Type: application/json" \
4 -d '{
5 "path": "persona",
6 "output_message": {
7 "content": "Well, you know, there is so much secrecy involved in government, folks, it\'s unbelievable. They don\'t want to tell you everything. They don\'t tell me everything! But about Roswell, it’s a very popular question. I know, I just know, that something very, very peculiar happened there. Was it a weather balloon? Maybe. Was it something extraterrestrial? Could be. I\'d love to go down and open up all the classified documents, believe me, I would. But they don\'t let that happen. The Deep State, folks, the Deep State. They’re unbelievable. They want to keep everything a secret. But whatever the truth is, I can tell you this: it’s something big, very very big. Tremendous, in fact.",
8 "role": "assistant"
9 },
10 "prompt_tokens": 100,
11 "output_tokens": 220,
12 "prompt_cost": 0.00001,
13 "output_cost": 0.0002,
14 "finish_reason": "stop",
15 "messages": [
16 {
17 "role": "user",
18 "content": "What really happened at Roswell?"
19 }
20 ],
21 "prompt": {
22 "model": "gpt-4",
23 "template": [
24 {
25 "role": "system",
26 "content": "You are {{person}}. Answer questions as this person. Do not break character."
27 }
28 ]
29 },
30 "created_at": "2024-07-19T00:29:35.178992",
31 "provider_latency": 6.5931549072265625,
32 "inputs": {
33 "person": "Trump"
34 }
35}'

Serialization (.prompt file)

Our .prompt file format is a serialized version of a model config that is designed to be human-readable and suitable for checking into your version control systems alongside your code. See the .prompt files reference reference for more details.

Format

The .prompt file is heavily inspired by MDX, with model and hyperparameters specified in a YAML header alongside a JSX-inspired format for your Chat Template.

Basic examples

1---
2model: gpt-4
3temperature: 1.0
4max_tokens: -1
5provider: openai
6endpoint: chat
7---
8<system>
9 You are a friendly assistant.
10</system>