Diego Hidalgo Soto
06/19/2024, 9:06 AM@task(retries=3, retry_delay_seconds=10, timeout_seconds=2 * 60 * 60)
def undefined_task_function(...):
pass
but for some reasons, I would like to increase the timeout_seconds
when it is called from the flow. How could I do that? If I do this:
@flow
def flow_function():
df = undefined_task_function(
some_parameters,
timeout_seconds=7000,
)
print(df)
flow_function()
it says the following:
prefect.exceptions.ParameterBindError: Error binding parameters for function 'undefined_task_function': got an unexpected keyword argument 'timeout_seconds'.
Nate
06/19/2024, 6:15 PMwith_options
here like this
In [1]: from prefect import flow, task
In [2]: import time
...:
...: @task
...: def no_timeout_by_default():
...: time.sleep(1000000000)
...:
...:
In [3]: @flow
...: def test():
...: no_timeout_by_default.with_options(timeout_seconds=2)() # call modified task
...:
In [4]: test()
13:14:41.311 | INFO | prefect.engine - Created flow run 'voracious-auk' for flow 'test'
13:14:42.012 | INFO | Task run 'no_timeout_by_default-0' - Created task run 'no_timeout_by_default-0' for task 'no_timeout_by_default'
13:14:44.300 | ERROR | Task run 'no_timeout_by_default-0' - Task run failed with exception TimeoutError('Scope timed out after 2.0 second(s).') - Retries are exhausted
13:14:44.302 | ERROR | Task run 'no_timeout_by_default-0' - Task run exceeded timeout of 2.0 seconds
13:14:44.420 | ERROR | Task run 'no_timeout_by_default-0' - Finished in state TimedOut('Task run exceeded timeout of 2.0 seconds', type=FAILED)
13:14:44.424 | ERROR | Flow run 'voracious-auk' - Flow run exceeded timeout of None seconds
13:14:44.575 | ERROR | Flow run 'voracious-auk' - Finished in state TimedOut('Flow run exceeded timeout of None seconds', type=FAILED)
Diego Hidalgo Soto
06/20/2024, 5:09 AM@flow
def genesys():
df = genesys_to_df(
"""parameters"""
).with_options(timeout_seconds=2)()
print(df)
genesys()
Diego Hidalgo Soto
06/20/2024, 5:10 AMtimeout_seconds
parameter is defined in the @task
decorator, the flow is running normally.Nate
06/20/2024, 2:11 PMwith_options
allows you to return a version of a task with modified settings, which sounds like what you wantDiego Hidalgo Soto
06/21/2024, 5:38 AMwith_options
...Nate
06/21/2024, 2:15 PMKevin Grismore
06/21/2024, 2:17 PMtask_function(params).with_options(options)()
when we're expecting task_function.with_options(options)(params)
Nate
06/21/2024, 2:20 PM@flow
def genesys():
df = genesys_to_df.with_options(timeout_seconds=2)(**parameters)
print(df)
genesys()
Nate
06/21/2024, 2:21 PMwith_options
is a @classmethod
that returns a Task
that you should then call as needed