Thread
#prefect-community
    Paco Ibañez

    Paco Ibañez

    3 months ago
    Hello! I have a flow in 2.0 with two tasks that I want to run sequentially. I am using a docker flow runner and my flow looks something like this:
    @flow
    def records_analysis_flow():
        ...
        future = ingest_csv(config)
        future.wait()
        future = analyze_records(ra_config)
        future.wait()
    I noticed somewhat inconsistent performance results. If I run the code in docker without prefect it takes around 154 seconds but when I run it in prefect it takes almost 900 seconds. However, if I explicitly set the task runner to sequential, then it takes ~ 180 seconds. Is my flow doing something wrong? Why does the concurrent task runner take so long? Thanks!
    Michael Adkins

    Michael Adkins

    3 months ago
    We’ll need more details about you flow/tasks, I’m surprised to see this behavior. If you can find out where the time is being spent, that’d be really helpful.
    (or provide a minimal example that I can profile)
    Is it still slow without those
    wait
    calls?
    Paco Ibañez

    Paco Ibañez

    3 months ago
    the first task is always super quick, the second one (which uses a language model) is the one that is really slow
    but if I set the task runner to sequential it performs way better
    @flow(task_runner=SequentialTaskRunner())
    def records_analysis_flow():
       ...