Paco Ibañez
06/14/2022, 2:34 PM@flow
def records_analysis_flow():
...
future = ingest_csv(config)
future.wait()
future = analyze_records(ra_config)
future.wait()
I noticed somewhat inconsistent performance results. If I run the code in docker without prefect it takes around 154 seconds but when I run it in prefect it takes almost 900 seconds. However, if I explicitly set the task runner to sequential, then it takes ~ 180 seconds. Is my flow doing something wrong? Why does the concurrent task runner take so long? Thanks!Zanie
06/14/2022, 2:45 PMwait
calls?Paco Ibañez
06/14/2022, 2:49 PM@flow(task_runner=SequentialTaskRunner())
def records_analysis_flow():
...