d

    Dean Magee

    1 year ago
    Hi All, I need to trigger the same flow but with 40+ different sets of parameters. Currently im doing the following...
    for unique_client in function_that_returns_list_of_40+_parameter_dicts():
        flow.run(unique_client)
    Surely there is a better way to run the same flow but with different parameters each time. Am I missing something?
    Chris White

    Chris White

    1 year ago
    With Prefect Core alone, managing large quantities of Flows with varying parameter sets is pretty clunky (as you’re seeing).
    flow.run
    is largely intended for one-off runs and local testing. This is the sort of situation the Prefect backends are designed to handle
    d

    Dean Magee

    1 year ago
    Thanks Chris. Is there a way to programatically trigger flows to start through the backend?
    Chris White

    Chris White

    1 year ago
    Yup definitely! The backend exposes an API that allows you to do most anything, including creating ad-hoc / on-demand flow runs with different parameter values (https://docs.prefect.io/orchestration/concepts/flow_runs.html for more details). For a worked example that programmatically triggering flows from AWS Lambda, check out this post (open in incognito if you need to): https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a And just in case, if you want to run your Flow on a well-defined schedule with varying parameter values, that’s possible too: https://docs.prefect.io/core/concepts/schedules.html#clocks