Adam Everington
10/20/2021, 8:35 AMwith Flow('test_flow',executor=LocalExecutor()) as flow:
df = sometask_that_returns_a_df()
count_of_records_in_df = len(df.index)
execute_task_2(count_of_records_in_df)
Obvs compilation fails because the return type of sometask is a functiontaskAnna Geller
with Flow()
constructor:
@task
def get_len_of_df(df):
return len(df.index)
with Flow('test_flow',executor=LocalExecutor()) as flow:
df = sometask_that_returns_a_df()
count_of_records_in_df = get_len_of_df(df)
execute_task_2(count_of_records_in_df)
Adam Everington
10/20/2021, 9:10 AMAdam Everington
10/20/2021, 9:11 AMAdam Everington
10/20/2021, 9:18 AMclass DTO:
def __init__(self,data:pd.DataFrame):
self.data=data
self.data_count=len(self.data.index)
had this as a return type of a task:
@task
def some_task->DTO:
thisDTO = DTO(data_frame)
return thisDTO
could i then access that within the flow like so:
with Flow('test_flow',executor=LocalExecutor()) as flow:
dto1 = some_task()
task2(dto1.data_count)
would I face the same issues there?Anna Geller
class DTO(Task):
def run(self, data, data_count):
# do sth
return data
Kevin Kho
cloudpickle
, which I think it should be.