Hi everyone! I'm using Prefect V1 and I'm having some issues with the timeout configuration. Our task is kicking off

ShellTask

which is written in Typescript. As part of the typescript script we call python script. The prefect timeout works if it's reached while typescript is executing, but the timeout doesn't work if the process is stuck inside the python script. Has anyone ran into the same issue before in the past? Any advice is appreciated, thank you in advance!

Hey Nace! Would you mind providing a minimal reproducible example, and the version of prefect that you're on?

I'm using Prefect

v1.3.1

, and here are the screenshots of all configs that I am able to reproduce the issue with:

Hey Nace, I've raised this to the team to see what could be happening here. I'll see if I can replicate on my end as well.

@Bianca Hoch do you have any update on this?

Hi Nace, we're going to take a closer look at this today. At first glance, it could be the case that Prefect is not monitoring the execution of the python script, which would explain why the timeout is not being applied during that timeframe. A potential solution could be to apply an additional timeout arg to handle the python script separately. Is the python script being called also a prefect flow?

The python script is not its own flow, but just a part of the typescript script that is being kicked off as a flow. Adding arguments to python scripts with timeouts would work in that case. However, I discovered that also if a command is stuck in a typescript (so when typescript is waiting on it), the timeout doesn't work. Does Prefect V2 handle timeouts in the same way? A little bit of background why this is currently an issue - we are planning to expend significantly in the next few months and we are still evaluating if Prefect is the right tool for us. Currently we don't have many flows yet, but we are still planning to reorganize them a little bit and add a few new ones. Currently the biggest issue we are having is that flows are stuck in a forever running state and preventing other flows to run, but runs for all our customers are crucial for us. We are also considering upgrading to Prefect V2 as part of the same effort (we still need to evaluate how much extra effort that would take though), and if there are no issues with timeouts in v2 that would be a huge motivating factor for us to try out v2 already at this point.

Timeouts are indeed handled differently between 1.0 and 2.0. In Prefect 2, you can implement timeouts at the flow-level by setting the timeout_seconds parameter in the

@flow

decorator. When the runtime of a flow exceeds the timeout, the flow is marked as failed. This would help mitigate the flow getting stuck in a running state.

That helps a lot, thanks! So migrating to V2 might help us avoid the existing issue. And if I understand correctly the goal is to eventually move everyone off V1, right? With that, V2 is more stable and more maintained? I'm really tempted to try to convince my team to migrate to V2 already now but we do have basically just one week to get these runs in order, so I am torn between fixing up and rearranging the current flows in v1 OR just starting a new environment with V2 and set that up. I know there are some docs on how to migrate, but any advice is really appreciated about if it's a smart idea to be doing the transition to v2 right now under tight time constraint

Totally understandable. Prefect 1 will be supported through at least June 2023. Prefect 2 will continue to be improved and maintained in the long term, so I would suggest incrementally migrating flows to 2.0 as soon as you are able.

Sweet, thank you so much for your help! That helps. I think I'll suggest moving the few scheduled runs to V2, hopefully the setup overhead (resources and all that) won't be too time consuming.

Absolutely, definitely reach out to us in slack if you have migration questions.

Sounds good, thank you! Thanks for all the support!!

Also our discourse is a great resource for migration questions (and much more).