Tim Galvin
06/22/2023, 11:47 AM@task
decorated function blocking when calling it as is, e.g.
@task
def square_num(a):
return a**2
@flow
def main_flow():
squares = [square_num(i) for i in range(4)]
here each time square_num
is called in the loop, it is not a PrefectFuture
that is stored, it is the evaluated number. To get the behaviour that is similar to the @dask.delayed
type decorated function, the .submit
method should be used.
I ask this question after mocking up a port of a dask
workflow over to prefect
. If the __call__
of a Task
instance behaved similar to its submit
method, the conversion would basically be as simple as replacing @delayed
with @task
and calling it a day, I mocked up a extra Task
keyword argument in my own prefect
fork called block_on_call
, and could tweak this __call__
vs .submit()
behaviour on a per @task
basis. Maybe it is because of my previous dask
exposure, but this kind of feels more natural to me, and kind of makes this toy porting problem pretty effortless.
Is it a case that the current Task.__call__
behaviour is to support a DAG-less type use case/style? Would there be much interest in a pull request of the above make sense/be useful, or am I missing the larger intent here? Does my rambling make sense?Ryan Peden
06/22/2023, 7:26 PM@task
decorator.Tim Galvin
06/23/2023, 1:47 AM@task
to a function and call the function as it is written without prefect involved? The net effect is the same, modulo the monitoring on the prefect server. My initial feeling would have been the concurrent processing that prefect provides would be more valued than the monitoring in a blocking type mode.
But I say all this as one user - and I really hope that I am not a painful one!
My little patch in my own test environment can revert this behaviour on a per-task basis, with the default behaviour being what is currently implement, i.e. The old behaviour is only used when @task(block_on_call=False)
which is not the default. I would be happy to submit a pull request for review if it makes senseTim Galvin
06/23/2023, 5:37 AM