Kaz
04/05/2020, 7:23 PMShellTask
doesn’t seem to work for me. I have an S3 sync that occasionally hang (working to figure that out), and as a temporary solution I was hoping to use a timeout + retry. This has solved the issue in the past. I’ve set the timeout to 600 (600 seconds, 10 minutes), and yet I’m getting a task that runs for 6 hours+ when the shell task hangs. On Cloud (Scheduler tier), I can see that my retry and retry delay parameters are set, but I’m not sure if there’s a way to check whether a timeout has been configured on a task. It’s a fairly simple task (top level, only 1 dependency, unmapped), and everything works just fine if I manually restart the task on the Cloud UI (I even see the retry count increment). I’m wondering if there are any additional inputs I needed to configure other than timeout + retry in order to get the timeout to work, or if this is an issue other people have seen before?. Thanks!Chris White
ShellTask
we certainly could. If this is important for your workflow we should open an issue on the repo to look into fixing this!Kaz
04/05/2020, 9:13 PMChris White
ShellTask
code but then replace these lines: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/shell.py#L95-L104 with this one line:
sub_process.wait(timeout=600)
you should be good