Hi all, Im learning the basics of Prefect and have a quick question. When I write this and register my flow to the cloud:
Copy code
with Flow("flow-a") as flow_a:
a()
b()
flow = Flow("flow-a-b", tasks=[a, b])
flow.register(project_name="tester")
Is the code written in function
a
and
b
saved on the cloud, or does the cloud refers to my system every time it needs to run the flow on an agent? If it does refer to my system, will it make a difference if the agent is running on a different system?
k
Kevin Kho
05/12/2022, 1:44 PM
We don’t see your code and data because of the hybrid model. Prefect doesn’t host any compute so all of the compute happens in your infrastructure
a
Anna Geller
05/12/2022, 1:45 PM
During the registration, you send metadata to Prefect Cloud - Prefect doesn't store your code or data, only metadata that points to the storage location telling where your flow code is stored.
will it make a difference if the agent is running on a different system?
it's the same for all agents
lastly, you may actually move the register part into main:
Copy code
with Flow("flow-a") as flow:
task_a = a()
b(upstream_tasks=[task_a])
if __name__ == "__main__":
flow.register(project_name="tester")
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.