Hi all. Having some issues with running multiple copies of the same flow (via FlowRunTask) simultaneously within a larger flow -- wondering if anyone has any advice. I setup a flow that triggers jobs in Azure Data Factory and another flow that executes a stored procedure in a MS SQL Server. Each of these requires a parameter to be passed in (ADF pipeline name or stored procedure name). I then create "master" flows that calls each of these flows specifying the parameter within the code. In one specific "master" flow, I call the ADF flow twice to trigger two separate runs passing different parameters in each. However, it appears as though the ADF flow is only being triggered once. While it appears as both flows ran independently, closer inspection into the actual task runs reveals the same run ID. Using LocalExecutor with Prefect Server. Perhaps I should run these in series instead of parallel? Perhaps I should update to StartFlowRun? Appreciate any advice and happy to share more info.

Hi @brett - the flow run task uses the active Flow Run ID as an

idempotency_key

to gracefully handle reruns / retries; this means that if you are triggering the same flow with this task, only one run will get created. To alter this behavior, you’ll need to provide your own idempotency key to the

run

method of your tasks. Note that you’ll want this key to change with each new run of your main Flow, so you might need to introduce a new task that generates keys for each run.

Thanks, @Chris White! Am I understanding this correctly? When the parent (master) flow runs, an

idempotency_key

is generated. That key is passed to each FlowRunTask. Because I'm calling the same child flow twice (despite receiving different parameters), both runs receive the same key. The worker agent completes one of these flow instances and then serves the cached result for the second run. So the solution would be to create two key generation tasks that sit as predecessors to the ADF flow runs? That would ensure a unique key is generated for each ADF FlowRunTask? FYI: This parent flow runs every 24 hours