https://prefect.io logo
Title
b

Ben Muller

02/25/2022, 3:52 AM
Hey, I am running a flow locally and have another flow deployed in the cloud. I loop over something and create multiple sub flows.
create_flow_run.run(
            flow_name="advanced_cleaning",
            project_name="modelling",
            run_name=f"data_update={data_update}",
            parameters=dict(data_update=data_update),
        )
When I run this it is called multiple times, but only one sometimes two flows are triggered in the cloud, is this a possible bug? Do run names need to be unique? This is being called from within a task fyi
I see there is a StartFlowRun also, but this gives me the same issues 🀷
in fact the loop, every time it calls the task to create a new flow run, returns the exact same flow run url 🀷
k

Kevin Kho

02/25/2022, 4:11 AM
This has to do with the idempotency key. These tasks use the context task run id as the default idempotency key. Multiple invocations will use the same key and only trigger one flow run, so you need to supply multiple idempotency keys to generate multiple runs
b

Ben Muller

02/25/2022, 4:28 AM
are there any examples of this? I dont really get what you are saying, where do I get the idempotency key from ? And why does it work fine when it is running from the cloud flow and not locally?
k

Kevin Kho

02/25/2022, 4:34 AM
You can just supply any string as the idempotency key so you can do something like datetime.now() as a string and it will trigger multiple runs
I guess this is an example but it’s the client not as task
It should behave the same for both in that 1 run is created
b

Ben Muller

02/25/2022, 5:18 AM
got it, thanks mate
Fixed the issue. Thanks @Kevin Kho πŸ™
πŸ‘ 1
Hey @Kevin Kho, this ran fine when triggering locally but when it runs on the schedule in the cloud, I get a
Error during execution of task: TypeError("create_flow_run() got an unexpected keyword argument 'idempotency_key'")
Locally I am using prefect
0.15.5
and my prefect core version is the same on the UI. Any ideas?
k

Kevin Kho

02/28/2022, 8:37 PM
That was added actually let me see the changelog
You need 0.15.8
b

Ben Muller

02/28/2022, 8:38 PM
ah ok, any breaking changes I should be aware of if I upgrade?
k

Kevin Kho

02/28/2022, 8:39 PM
From 0.15.5 to 0.15.8 should be absolutely nothing
b

Ben Muller

02/28/2022, 8:39 PM
incredible
k

Kevin Kho

02/28/2022, 8:40 PM
You seen to be up super late?
b

Ben Muller

02/28/2022, 8:40 PM
it is 7.40am πŸ˜†
You can never get my timezone right lol
k

Kevin Kho

02/28/2022, 8:41 PM
Ah lol cuz you normally ping later in the day
b

Ben Muller

02/28/2022, 8:41 PM
yeah, I usually start work, then get stuck and you save me lol
πŸ˜† 1