https://prefect.io logo
Title
n

Nace Plesko

10/03/2022, 7:47 PM
Hi, I'm using Prefect v1 and I'm having trouble with configuring timeouts for
ShellTasks
. Setting timeout like it says in the documentation doesn't have any effect, the task doesn't get terminated after 10 seconds in this case. Is that a bug in Prefect or is it an issue on my part and I'm somehow misconfiguring
ShellTask
? I really appreciate any help on this and thank you in advance!
1
n

Nate

10/03/2022, 8:11 PM
Hi @Nace Plesko -
ShellTask
seems to be behaving as expected for me on
prefect==1.2.4
from prefect import Flow
from prefect.tasks.shell import ShellTask

with Flow('test shell task timeout') as flow:
    cmds = ['sleep 4; echo "done"', 'sleep 6; echo "done"']
    
    ShellTask(timeout=5, stream_output=True).map(command=cmds)
    
if __name__ == '__main__':
    flow.run()
yields
❯ python shelltasktimeout.py
[2022-10-03 15:10:02-0500] INFO - prefect.FlowRunner | Beginning Flow run for 'test shell task timeout'
[2022-10-03 15:10:02-0500] INFO - prefect.TaskRunner | Task 'ShellTask': Starting task run...
[2022-10-03 15:10:02-0500] INFO - prefect.TaskRunner | Task 'ShellTask': Finished task run for task with final state: 'Mapped'
[2022-10-03 15:10:02-0500] INFO - prefect.TaskRunner | Task 'ShellTask[0]': Starting task run...
[2022-10-03 15:10:07-0500] INFO - prefect.ShellTask | done
[2022-10-03 15:10:07-0500] INFO - prefect.TaskRunner | Task 'ShellTask[0]': Finished task run for task with final state: 'Success'
[2022-10-03 15:10:07-0500] INFO - prefect.TaskRunner | Task 'ShellTask[1]': Starting task run...
[2022-10-03 15:10:13-0500] INFO - prefect.TaskRunner | Task 'ShellTask[1]': Finished task run for task with final state: 'TimedOut'
[2022-10-03 15:10:13-0500] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
I'm not immediately sure what could be wrong with your configuration 🤔
n

Nace Plesko

10/03/2022, 8:14 PM
Thanks for testing it out! Maybe it is in the version. I have
prefect==1.2.2
, but in the UI under "Flow Overview" I see
Prefect Core Version 1.3.1
n

Nate

10/03/2022, 8:24 PM
hmm I checked 1.3.1 both locally and on cloud and they also appear to behave as expected for me. I wonder if my snippet behaves as expected on your machine?
n

Nace Plesko

10/03/2022, 8:32 PM
Whaaat, it was about versioning!!! Huh, thank you! What happened was that we didn't have Prefect version specified in requirements.txt and I think we got then both 2.X and 1.X and that was confusing it. Cleaning up all packages and reinstalling requirements fixed it. Thank you!!
n

Nate

10/03/2022, 8:32 PM
awesome - great to hear 😄
🎉 1
n

Nace Plesko

10/04/2022, 2:08 PM
Hm, last night the scheduled runs again got stuck in running state for over 9 hours, even though I set the timeout to 7200 seconds which is 2 hours
n

Nate

10/04/2022, 2:39 PM
hmm - that sounds like a different problem. If this problem persists, can you open a new thread and describe what you see there?
👍 1