Flows

Flows allow you to trace each step of an AI pipeline, such as a Retrieval-Augmented Generation (RAG) pipeline or an agent. This tracing enables you to evaluate the performance of your AI pipeline as a whole, and to identify issues that may be occurring in specific steps.

Flow Logs

A Flow Log captures interactions between multiple Humanloop Files, such as Prompts, Tools, Evaluators, and even other Flows. Other Files’ Logs represent single interactions with their File. A Flow Log links multiple Logs to create a trace of an entire feature or process.

Linking Logs to Flow Logs

A Flow Log can link Logs from multiple other Files into a single, hierarchical trace. All Logs have a trace_parent_id property. Through it, a Log can be linked directly to the Flow Log or under any other Log in the trace. This allows Flows to represent complex workflows composed of multiple Files.

Flow versioning

For Flows, versioning is determined by the attributes field — an arbitrary JSON object that acts as a manifest of the instrumented feature.

Monitoring and completion

A Flow Log should be marked as completed when all Logs have been added to the trace. This makes the Flow Log available to its monitoring Evaluators. Evaluators applied to Flows have access to all Logs inside the trace: Use the children attribute of the log to access the nested Logs.

Flow-level metrics

The start time and end time of a Flow Log are determined by the Logs within: when adding a Log to the trace, the Flow Log’s start time and end time are updated to reflect the earliest and latest points in time respectively. Attributes related to cost and tokens sum up the values from all Logs in the trace.