Avi A
07/15/2020, 1:21 PMdict
, prefect handles it by planting the getItem
task in the DAG:
dict_output = generate_dict()
a = task_a(dict_output['a'])
b = task_b(dict_output['b'])
How would you recommend working with dataclasses? The following piece of code won’t work, naturally (since a task doesn’t have these attributes)
@dataclass
def MyClass:
a: int
b: int
...
...
dataclass_output = generate_dataclass()
a = task_a(dataclass_output.a)
b = task_b(dataclass_output.b)
c = another_task(dataclass_output)
Jeremiah
07/15/2020, 1:26 PMGetItem
tasks (see here) - you could replicate this with a GetAttribute
task that basically runs lambda x: x.attr
. Adding sugar for that might be tricky, because we don’t want to allow magic attribute access to the task class. Maybe something like dataclass_output.output.a
or something? Just idle thoughts.Avi A
07/15/2020, 1:29 PMdataclass_output.output
allows direct interaction with a `Task`’s result?Jeremiah
07/15/2020, 1:29 PMGetAttribute
task like:
@task
def get_attribute(task, attr):
return getattr(task, attr)
a = get_attribute(dataclass_output, 'a')
Avi A
07/15/2020, 1:31 PMJeremiah
07/15/2020, 1:31 PMAvi A
07/15/2020, 1:33 PMGetItem
doesn’t persist even though Result
is defined on the flow, which makes a lot of sense b/c it just extracts data from existing data. How can I do the same for this task? would it help to just set target=None
?Jeremiah
07/15/2020, 1:34 PMGetItem
task AFAIK[]