Michael Adkins
08/26/2019, 3:29 PMChris White
08/26/2019, 4:35 PMJoe Schmid
08/26/2019, 7:21 PMMichael Adkins
08/26/2019, 8:07 PMChris White
08/26/2019, 8:29 PMfrom prefect import Task
class MyTask(Task):
def __init__(self, task_a_kwargs, task_b_kwargs, task_c_kwargs, **kwargs):
self.composed_tasks = [TaskA(**task_a_kwargs), TaskB(**task_b_kwargs)]
super().__init__(**kwargs)
def run(self):
return_a = self.composed_tasks[0]()
return_b = self.composed_tasks[1](return_a)
Michael Adkins
08/26/2019, 9:31 PMChris White
08/26/2019, 9:39 PMMyTask
task, and only it will be submitted to an executor (as a single future). While we are importing / calling other “tasks”, we’re directly calling their run
methods in our logic, so effectively all we’re doing is just re-using their boilerplate code.Michael Adkins
08/26/2019, 9:54 PMChris White
08/26/2019, 9:56 PMalways_run
trigger. I think in that case the Prefect states would better reflect your actual intent.Michael Adkins
08/26/2019, 10:07 PMWorkTask
to run within the pod created by PodCreation
but there's not really a clear way to do thatChris White
08/26/2019, 10:11 PM<mailto:chris@prefect.io|chris@prefect.io>
and we can try to set something up