Kaz
04/05/2020, 7:23 PMShellTask
doesn’t seem to work for me. I have an S3 sync that occasionally hang (working to figure that out), and as a temporary solution I was hoping to use a timeout + retry. This has solved the issue in the past. I’ve set the timeout to 600 (600 seconds, 10 minutes), and yet I’m getting a task that runs for 6 hours+ when the shell task hangs. On Cloud (Scheduler tier), I can see that my retry and retry delay parameters are set, but I’m not sure if there’s a way to check whether a timeout has been configured on a task. It’s a fairly simple task (top level, only 1 dependency, unmapped), and everything works just fine if I manually restart the task on the Cloud UI (I even see the retry count increment). I’m wondering if there are any additional inputs I needed to configure other than timeout + retry in order to get the timeout to work, or if this is an issue other people have seen before?. Thanks!Chris White
04/05/2020, 7:38 PMShellTask
we certainly could. If this is important for your workflow we should open an issue on the repo to look into fixing this!Kaz
04/05/2020, 9:13 PMChris White
04/05/2020, 10:03 PMShellTask
code but then replace these lines: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/shell.py#L95-L104 with this one line:
sub_process.wait(timeout=600)
you should be good