Channels
Greetings, friends! Anyone have experience with ad...
# ask-communityc
Chris Leber
06/02/2021, 8:52 PMGreetings, friends! Anyone have experience with adding timeouts to the child tasks spawn from mapping? With a single task, I have no problem adding timeout arg to @task decorator. But when I do the same for a task that is mapped, there seems to be no effect.
👀 1
k
Kevin Kho
06/02/2021, 8:55 PM
Hi @Chris Leber! I’ll look into this for you.
c
Chris Leber
06/02/2021, 8:55 PMThanks @Kevin Kho!! I very much appreciate it.
k
Kevin Kho
06/02/2021, 8:57 PM
I have this code snippet and it seems to work:
Copy code
from prefect import Flow, task
import time
@task(timeout=2)
def abc(x):
time.sleep(3)
return x
with Flow(
"mapped-failures",
) as flow:
map_results = abc.map([1,2,3,4])
flow.run()Kevin Kho
06/02/2021, 8:58 PM
Is yours failing on Cloud/Server? or also on LocalExecution? Is this representative of what you have?
c
Chris Leber
06/02/2021, 9:00 PMFailing on Cloud, and yes representative of what I am doing 🤔 quite perplexing
Chris Leber
06/02/2021, 9:07 PMI wonder if it has to do with the nature of the task, as it seems to be causing all sorts of strangeness. The task involves using pycurl for downloading files from a database. Sometimes the files are available for downloading immediately, sometimes they are 'pending' as they need to be retrieved from a tape archive (can take up to 24 hours).
Other strangeness: I cancelled a few flows that were getting stuck on one of these 'pending' files - hours later, the flows do not report that they have been "cancelled". Instead, they still report that they are "cancelling", and a look at their tasks shows that the task in question is indeed still "running".
k
Kevin Kho
06/02/2021, 9:30 PM
Let me test this on cloud real quick
Kevin Kho
06/02/2021, 9:36 PM
So cloud is working on my end with the LocalDaskExecutor. Are you on a DaskExecutor?
c
Chris Leber
06/02/2021, 9:46 PMCurrently not explicitly selecting an executor
Chris Leber
06/02/2021, 9:49 PMSo whatever the default executor is
Chris Leber
06/02/2021, 9:54 PMTrying LocalDaskExecutor currently
k
Kevin Kho
06/02/2021, 10:04 PM
This is odd. I’ll probably have to look at it a bit later
c
Chris Leber
06/02/2021, 10:44 PMLocalDaskExecutor allowed timeouts to occur! What is the default Executor? Still strange that the default did not allow
k
Kevin Kho
06/03/2021, 12:50 AM
LocalExecutor, which runs things sequentially
Kevin Kho
06/03/2021, 12:50 AM
Maybe it didn’t feel like a timeout because of the sequential execution of the mapped tasks?