Paweł Biernat
10/08/2024, 8:26 PMtask.serve()
functionality, let's say task_a and task_b. From what I gathered from the docs, to execute a task I have to
from module_a import task_a
result_a = task_a.delayed(1)
and the same with task_b
. Now, if I want to run an equivalent of result = task_b(task_a)
, I'd have to import both tasks and run
from module_a import task_a
from module_b import task_b
result_a = task_a.delay(1).wait().result() # not sure about the wait().result()
result_b = task_b.delay(result_a).wait().result()
But what if I can't import both modules (e.g. due to dependency conflicts, or different runtime environments)? What's the intended pattern to connect two relatively independent tasks?
With the flows this is easy, because I can write a parent flow that internally runs run_deployment("flow_a")
and then retrieve the result and pass it to run_deployment("flow_b")
. The only import I have to do is the prefect itself, the parent flow doesn't care about implementation details of the subflows, once they are served/deployed.Nate
10/08/2024, 8:31 PM@task
def a(): pass
@task
def b(): pass
@task
def to_serve(inputs):
b(a())
to_serve.serve()
• background tasks might not suit your use case, since generally they're nice for cases where you want to scale instances of the tasks a lot horizontally (since unlike flows, its websockets and you dont need a unique port) and they perform side affects (like send an email on user signup)
• it may end up being easier to use flows as you mention
• remember that you can use state hooks here to emit custom events to signal downstream deployments if need bePaweł Biernat
10/08/2024, 9:00 PMPaweł Biernat
10/08/2024, 9:00 PMPaweł Biernat
10/09/2024, 8:19 AMPaweł Biernat
10/09/2024, 8:22 AM