https://prefect.io logo
#prefect-community
Title
# prefect-community
n

Nikolaus Landgraf

05/31/2022, 12:38 PM
Hi, we migrated from prefect 1.0 to prefect 2.0. With this migration we are experiencing huge performance issues running the same flows as before. Interestingly, when keyboard interrupting a prefect flow on the CLI the task continues to run - with better performance. It seems this keyboard interrupt exits a prefect wrapper that is responsible for the performance issues. Attached is a screenshot of the error code when interrupting a prefect flow. Anybody experiences similar performance issues after the migration?
This issue happened when using the "Concurrent Taskrunner". Switching to the "Sequential Taskrunner" seems to work better
k

Kevin Kho

05/31/2022, 2:23 PM
Is your code simple enough to share or could you make an example around this for us to take a look at?
n

Nikolaus Landgraf

05/31/2022, 5:14 PM
We have created 1 flow which contained 1 parameter, 1 normal function and several (very similar) tasks with dependencies. The tasks followed all the same scheme (a simple for loop with DB requests). As described, this caused issues when using the sequential task runner. It works a lot better (by a factor of 4 faster) when running it with the sequential taskrunner.
k

Kevin Kho

05/31/2022, 5:31 PM
Thank you! The engineer responsible for this is out this week, but I’ll open an issue
👍 1
4 Views