@Marvin Does prefect have a builtin solution for having a single correlation id across the whole lifecycle of a run and its subflows?
m
Marvin
03/19/2024, 1:33 PM
Prefect treats subflow runs similarly to normal flow runs, meaning subflows start as new tasks within the parent flow and there is a full representation of the subflow run in the backend. However, the search did not yield results explicitly mentioning a built-in solution for using a single correlation ID for the entire lifecycle of a run and its subflows.
For detailed and latest information, you may wish to explore the concepts of flows and global coordination within Prefect documentation:
- Composing flows
- Overview - How Prefect Works
- Prefect Global Coordination Plane
If you need to implement something more sophisticated (like a custom correlation ID logic), you might have to extend the given functionality or use additional tooling to track the lifecycle.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.