haf
09/13/2021, 12:23 PMnout
to the task decorator/constructor, or provide a Tuple
return-type annotation to your task.` but I can't use nout
because I don't know the result size and it is not a good fit for Tuple (since it's a list of tuple)emre
09/13/2021, 12:35 PMn1, n2 = MyTask()
As you said, this is only useful if you know your output length in advance. Check your code whether if you are doing something similar.
Otherwise, there is nothing wrong about returning lists. From there, you can either pass the entire list to downstream tasks, or use mapping to iterate over each list element and create a downstream task per element.haf
09/13/2021, 12:36 PMlist( ... )
)haf
09/13/2021, 12:36 PMhaf
09/13/2021, 12:37 PMhaf
09/13/2021, 12:37 PMhaf
09/13/2021, 12:37 PMset
haf
09/13/2021, 12:38 PMhaf
09/13/2021, 12:38 PMhaf
09/13/2021, 1:09 PMtask(lambda as, xs: set(as).difference(xs), name="intersection")(metrics, results)
to wrap the metrics and results in the monadic task value.emre
09/13/2021, 1:09 PMemre
09/13/2021, 1:12 PMset
is not a prefect task. result_eligible_apps
is actually a Task
object that represents a tasks result. If you call a barebones set
over a task, python tries to iterate the task object and the error above happens.haf
09/13/2021, 1:13 PMrun
function to return prefect.Task[TRes]
where TRes is the actually non-wrapped function's return type?haf
09/13/2021, 1:14 PMcontextmanager
annotations does thisemre
09/13/2021, 1:14 PMtask1 + task2
is resolved as AddTask(task1, task2)
by prefect. But resolving arbitrary func calls like set(task1)
to a prefect task is not possible at this pointhaf
09/13/2021, 1:15 PMemre
09/13/2021, 1:16 PMhaf
09/13/2021, 1:17 PMhaf
09/13/2021, 1:17 PMIn general, users will not instantiate these tasks by hand; they will automatically be...
emre
09/13/2021, 1:18 PM{task1}
would get resolved into a Set(task1)
task definition, in this sense they are implicit. calling pythons default set
function cannot be detected by prefect howeverhaf
09/13/2021, 1:18 PMdef handle(xs: Set[Thing]) -> None:
I would have gotten the set of things automatically cast?emre
09/13/2021, 1:24 PM@task
def ret1():
return 1
@task
def ret2():
return 2
@task
def print_content(x):
print(x)
with Flow("ListTest") as f:
print_content([ret1, ret2])
emre
09/13/2021, 1:25 PM[]
operator created a list of 2 prefect tasks. and prefect, while building the flow, understood that the output of these two tasks should be appended in a list. And inserted a List
task to facilitate this.haf
09/13/2021, 1:26 PMhaf
09/13/2021, 1:26 PMemre
09/13/2021, 1:29 PMlist
call however would try to iterate over tasks, because the list
function assumes its inputs are iterable. You will simply get the above error, even before building a python list of tasks. Because task objects are not iterable.