m

    Minakshi

    2 years ago
    Is there a way to run multiple flows using prefect core? this doesn't seem to work. It only starts flow for the first dataset and then continues after the specified interval.
    for dataset in dataset_config['datasets']:
    print('starting flow for dataset' + dataset['dataset_name'])
    flow.run(dataset=dataset['dataset_name'])  # runs this flow on its schedule
    j

    josh

    2 years ago
    Hi @Minakshi each flow run needs to occur in its own process so either you should be calling
    .run
    for each dataset individually in its own session or take a look into using the server for deploying multiple flow runs across multiple processes https://docs.prefect.io/orchestration/server/overview.html
    m

    Minakshi

    2 years ago
    got it thanks!