Thread
#prefect-community
    Ben Muller

    Ben Muller

    7 months ago
    Hey, I am running a flow locally and have another flow deployed in the cloud. I loop over something and create multiple sub flows.
    create_flow_run.run(
                flow_name="advanced_cleaning",
                project_name="modelling",
                run_name=f"data_update={data_update}",
                parameters=dict(data_update=data_update),
            )
    When I run this it is called multiple times, but only one sometimes two flows are triggered in the cloud, is this a possible bug? Do run names need to be unique? This is being called from within a task fyi
    I see there is a StartFlowRun also, but this gives me the same issues 🀷
    in fact the loop, every time it calls the task to create a new flow run, returns the exact same flow run url 🀷
    Kevin Kho

    Kevin Kho

    7 months ago
    This has to do with the idempotency key. These tasks use the context task run id as the default idempotency key. Multiple invocations will use the same key and only trigger one flow run, so you need to supply multiple idempotency keys to generate multiple runs
    Ben Muller

    Ben Muller

    7 months ago
    are there any examples of this? I dont really get what you are saying, where do I get the idempotency key from ? And why does it work fine when it is running from the cloud flow and not locally?
    Kevin Kho

    Kevin Kho

    7 months ago
    You can just supply any string as the idempotency key so you can do something like datetime.now() as a string and it will trigger multiple runs
    I guess this is an example but it’s the client not as task
    It should behave the same for both in that 1 run is created
    Ben Muller

    Ben Muller

    7 months ago
    got it, thanks mate
    Fixed the issue. Thanks @Kevin Kho πŸ™
    Hey @Kevin Kho, this ran fine when triggering locally but when it runs on the schedule in the cloud, I get a
    Error during execution of task: TypeError("create_flow_run() got an unexpected keyword argument 'idempotency_key'")
    Locally I am using prefect
    0.15.5
    and my prefect core version is the same on the UI. Any ideas?
    Kevin Kho

    Kevin Kho

    6 months ago
    That was added actually let me see the changelog
    You need 0.15.8
    Ben Muller

    Ben Muller

    6 months ago
    ah ok, any breaking changes I should be aware of if I upgrade?
    Kevin Kho

    Kevin Kho

    6 months ago
    From 0.15.5 to 0.15.8 should be absolutely nothing
    Ben Muller

    Ben Muller

    6 months ago
    incredible
    Kevin Kho

    Kevin Kho

    6 months ago
    You seen to be up super late?
    Ben Muller

    Ben Muller

    6 months ago
    it is 7.40am πŸ˜†
    You can never get my timezone right lol
    Kevin Kho

    Kevin Kho

    6 months ago
    Ah lol cuz you normally ping later in the day
    Ben Muller

    Ben Muller

    6 months ago
    yeah, I usually start work, then get stuck and you save me lol