@Marvin How to run tasks using the map function synchronise?
It seems that the default behaviour is to run all the mapped tasks async, without waiting for them
m
Marvin
05/30/2024, 8:01 AM
It seems there isn't a direct method to run mapped tasks synchronously in Prefect. However, you can enforce ordering among tasks by using the
wait_for
keyword. Here's a relevant excerpt and code example from the documentation:
Copy code
python
from prefect import flow, task
@task
def task_1():
pass
@task
def task_2():
pass
@flow
def my_flow():
x = task_1()
# task_2 will wait for task_1 to complete
y = task_2(wait_for=[x])
This method can be adapted to your mapped tasks to ensure they run sequentially, by inserting dependencies explicitly or using a similar approach in a loop.
For additional information, you can refer to the documentation links:
- Tasks
- Map Function
- FAQ
If you need more detailed control or a specific pattern, please let me know!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.