Prompt Management
How to manage and version your Prompt and Tools on Humanloop
Prompts are the fundamental interaction method for large language models (LLMs). They define the instructions and parameters that guide the model’s responses for a specific task. In Humanloop, Prompts are managed with version control, allowing you to track changes and improvements over time.
Each change in any of the following properties creates a new version of the Prompt:
- the template such as
Write a song about {{topic}}
. For chat models, your template will contain an array of messages. - the model e.g.
gpt-4o
- all the parameters to the model such as
temperature
,max_tokens
,top_p
etc. - any tools available to the model
Creating a Prompt
You can create a Prompt explicitly in the Prompt Editor or via the API.
New prompts can also be created automatically via the API if you specify the Prompt’s path
(its name and directory) while supplying the Prompt’s parameters and template. This is useful if you are developing your prompts in code and want to be able to version them as you make changes to the code.
When to create a new Prompt
You should create a new Prompt for every different ‘task to be done’ with the LLM. For example each of these tasks are things that can be done by an LLM and should be a separate Prompt File: Writing Copilot, Personal Assistant, Summariser, etc.
We’ve seen people find it useful to also create a Prompt called ‘Playground’ where they can free form experiment without concern of breaking anything or making a mess of their other Prompts.
Versioning
A Prompt will have multiple versions as you experiment with different models, parameters, or templates. However, all versions should perform the same task and generally be interchangeable with one another.
By versioning your Prompts, you can track how adjustments to the template or parameters influence the LLM’s responses. This is crucial for iterative development, as you can pinpoint which versions produce the most relevant or accurate outputs for your specific use case.
As you edit your prompt, new versions of the Prompt are created automatically. Each version is timestamped and given a unique version ID which is deterministically based on the Prompt’s contents. For every version that you want to “save”, you commit that version and it will be recorded as a new committed version of the Prompt with a commit message.
Serialization (.prompt
file)
Our .prompt
file format is a serialized version of a model config that is designed to be human-readable and suitable for checking into your version control systems alongside your code. See the .prompt files reference reference for more details.
Format
The .prompt file is heavily inspired by MDX, with model and hyperparameters specified in a YAML header alongside a JSX-inspired format for your Chat Template.