Netanel Malka
02/12/2024, 12:53 PMMarvin
02/12/2024, 12:55 PM@task
.
2. Create a flow: You define a flow by using the with Flow("flow_name") as flow:
syntax.
3. Add tasks to the flow: Within the flow context, you can add tasks and define their order by setting dependencies between them. You will do this by calling the tasks like normal Python functions.
4. Set the flow's logic: Once your preparatory tasks are in the flow, you can continue to define the main logic of your flow by adding more tasks and specifying the execution order using dependencies.
5. Execute the flow: You can run the flow manually, through external schedulers, or by deploying and scheduling it on Prefect Cloud or a local Prefect server.
Here's an example of how this might look in code:
python
from prefect import Flow, task
@task
def preparatory_task_1():
# Code for the first task before the flow's main logic
pass
@task
def preparatory_task_2():
# Code for the second task before the flow's main logic
pass
@task
def main_flow_logic():
# Code for the main flow logic
pass
with Flow("My Flow") as flow:
task_1_output = preparatory_task_1()
task_2_output = preparatory_task_2(upstream_tasks=[task_1_output])
main_flow_logic(upstream_tasks=[task_2_output])
# Now, you can run the flow by invoking `flow.run()` or scheduling it as needed.
Keep in mind that tasks must be called from within a flow and not from other tasks. The order is determined by dependencies, and Prefect handles the task orchestration for you once your flow is running.Netanel Malka
02/12/2024, 1:15 PM