Pin the specified Evaluatee.
Pinned Evaluatees are always displayed in the Evaluation Overview, and serve as the baseline for comparison with other Evaluatees.
Unique identifier for Evaluation.
Unique identifier for the File Version. If provided, none of the other fields should be specified.
Path identifying a File. Provide either this or file_id
if you want to specify a File.
Unique identifier for the File. Provide either this or path
if you want to specify a File.
Name of the Environment a Version is deployed to. Only provide this when specifying a File. If not provided (and a File is specified), the default Environment is used.
Unique identifier for the batch of Logs to include in the Evaluation Report.
Whether the Prompt/Tool is orchestrated by Humanloop. Default is True
. If False
, a log for the Prompt/Tool should be submitted by the user via the API.
Unique identifier for the Evaluation. Starts with evr
.
The Dataset used in the Evaluation.
The Prompt/Tool Versions included in the Evaluation.
The Evaluator Versions used to evaluate.
The current status of the Evaluation.
"pending"
: The Evaluation has been created but is not actively being worked on by Humanloop."running"
: Humanloop is checking for any missing Logs and Evaluator Logs, and will generate them where appropriate."completed"
: All Logs an Evaluator Logs have been generated."cancelled"
: The Evaluation has been cancelled by the user. Humanloop will stop generating Logs and Evaluator Logs.Name of the Evaluation to help identify it. Must be unique among Evaluations associated with File.
Unique identifier for the File associated with the Evaluation.
URL to view the Evaluation on the Humanloop.