Hi Everyone, I am learning Prefect 2.0 from 10 day...
# prefect-getting-started
p
Hi Everyone, I am learning Prefect 2.0 from 10 days ago, so can you share any poc or tips that will helpful for to learn Prefect Thanks in advance
āœ… 1
n
hi @Praveen Gupta - welcome! I would get started here with the tutorial, it'll go through the need-to-know concepts. here's a quick script that will build you a deployment / get you started
Copy code
from prefect import flow

@flow(log_prints=True)
def my_flow(x: int, y: str = "Marvin", z: bool = True):
   print(x, y, z)

if __name__ == "__main__":
   my_flow.serve("my-first-deployment")
you can always ask @Marvin for help in #C04DZJC94DC as well šŸ™‚ marvin please say hello and give a concise, code-first tldr on prefect. show how to use
.serve
m
Hello šŸ‘‹ Prefect TL;DR: Prefect is a workflow management system that simplifies the creation, scheduling, and monitoring of complex workflows. Designed for modern infrastructure, it is an open source platform that provides a robust set of tools to build, run, and track data pipelines. For details, you can check out their website, here. Using .serve in Prefect: Prefect's
.serve
method allows you to deploy and schedule your flows with ease. Here's a concise code snippet demonstrating its usage:
Copy code
python
from prefect import flow

@flow
def my_flow():
    # Your flow logic here
    pass

if __name__ == "__main__":
    my_flow.serve(name="my_flow_deployment", interval=3600)
By executing this script, a flow named
my_flow_deployment
is scheduled to run every hour. It's worth noting that flows scheduled with this method run on the machine where the script is executed. For more details, you can reference the blog post here.
p
Hi Nate and thanks for your help and yes I am struggling in Prefect and yeah for starters for new learning it happens. There is one more thing I want to ask you, is it possible to learn and Master Prefect in 2 months because my mentor said to master in two months.
n
šŸ™‚ you can get a good set of skills under your belt in 2 months for sure! I wouldn't worry so much about mastery personally, just that you're enabling yourself to do what you want / need to do! feel free to reach out with specific technical questions that come up along your journey
p
Thanks Nate
j
I think you should go back to your mentor and ask what they mean by "master in two months" -- that's a really vague learning goal versus something like, "demonstrate your understanding of prefect by creating a prefect flow that has F1, F2, F3 features and runs on T1, T2 or T3 type infrastructure"
p
Yes @Janet Carson there is a question but expect that every thing is kinda vague like read docs all time cause there is not much tutorial on YouTube so question goes like this """ How to run two flows flow1 and flow2 and they are scheduled like flow1 is scheduled at 3:00 and flow2 is scheduled at 3:30 Condition- now considering something like flow2 is dependent on flow1 but my issue arrived when due some unknown reason flow1 end up running until 3.40 but flow2 is dependent on flow1 so I want if condition like this occurs sometimes the flow2 needs to be delayed 10 minutes or more but run after only and only when flow1 is complete it's run or come terminal state of finished. Please tell me how to do it? """
Hey @Nate , tell me pls why it won't get timeout even the time given for running the flow is exceeded """ from prefect import flow, task import time @flow(name="flow-for-timeout", timeout_seconds=10) def pls_get_timeout(): print("flow is started running") time.sleep(20) print("I won't time out easily buddy") if name == "__main__": pls_get_timeout() """
n
to answer your question about chaining flows, you can use
run_deployment
in an
on_completion
hook like
as for the timeout, I am getting
TimeoutError
as I would expect when I run your code
p
Hey @Nate thanks for help but why in my Prefect Cloud account, not getting timed out why it run successfully, why not raise timeout exception error is this cloud problem or something or UI problem ???
@Nate Also for second question deployment with scheduling I want to schedule both deployment like 3.00 and 3.30 just want to paused second while it running even it second flow got terminal state crashed that's okay, wait I wrote a code could you pls review it and make some changes for creating dependencies of flow2 on flow1,
n
i would only put a schedule on the first one (
greetings
) and then refer to my message above about using
on_completion
to trigger downstream work (like
Salutation
)
p
yes I understand that event hook on_completion perform this function not "flow" could do for flow like for task use we use my_task(wait_for=[my_other_task])
@Nate would pls edit the code and add flow2 dependency on flow1
n
there is no
wait_for
for flows, instead to create dependencies you have a couple options: • run_deployment as I've mentioned from a state hook, or you can call it from a
@task
in a parent flow
Copy code
@task
def run_my_deployment(**kwargs) -> FlowRun:
   return run_deployment(**kwargs)

@flow
def parent(a_kw: dict, b_kw: dict):
      flow_run_a = run_my_deployment(**a_kw)
      run_my_deployment(**b_kw, wait_for=[flow_run_a])
• or put a
DeploymentTrigger
on the downstream flow so that it just runs when the upstream deployment sees a
prefect.flow-run.Completed
event (my favorite option)
p
@Nate good things atleast it serve the priorities well if I have to choose second way I like to do instead of task wait+for
but what about timeout of flow it also won't work even after I delete whole workspace still it is successful
@Nate pls look into screenshots
what to do??
n
hi @Praveen Gupta - im sorry i dont have time to look into this right now. please feel free to ask your timeout question in #CL09KU1K7 where others can help
p
@Nate that okay and thanks for your help
šŸ‘ 1