Noah Holm
04/17/2024, 2:36 PMMarvin
04/17/2024, 2:36 PMCronSchedule
in a Prefect deployment. Let's try searching the Prefect documentation to find a relevant example or guide on how to use CronSchedule
in a Prefect 2.0 deployment.
Certainly! Below is an example of how to use a CronSchedule
in a Prefect deployment. Let's assume you have a flow defined already and want to set a schedule for your deployment. You can do so with CronSchedule
like this:
python
from prefect.deployments import Deployment
from prefect.orion.schemas.schedules import CronSchedule
# let's assume you have a flow function called `my_flow` defined elsewhere
cron_schedule = CronSchedule(
cron="0 0 * * *", # This cron expression means once a day at midnight
timezone="America/Chicago"
)
deployment = Deployment.build_from_flow(
flow=my_flow,
name="etl-deployment",
schedule=cron_schedule
)
deployment.apply()
# Now your flow will be scheduled to run once a day at midnight.
Here is how the cron schedule would be expressed in a prefect.yaml
file:
yaml
deployments:
# other deployment config
schedules:
- cron: "0 0 * * *"
timezone: "America/Chicago"
active: true
Please note that worker-based deployments are the recommended approach in Prefect 2.0, and using the prefect.yaml
file allows you to specify deployments and schedules.
You can find more information about schedules in the Prefect documentation.Noah Holm
04/17/2024, 2:44 PMprefect.server.schemas.schedules.CronSchedule
as well as prefect.client.schemas.schedules.CronSchedule
. Which should you use?Marvin
04/17/2024, 2:44 PMCronSchedule
from the path prefect.orion.schemas.schedules
. The prefect.server.schemas.schedules
import path you mentioned is more closely associated with Prefect 1.0, and prefect.client.schemas.schedules
seems to be another variant that isn't mentioned in the current Prefect 2.0 schedule documentation.
The relevant import for Prefect 2.0 would be:
python
from prefect.orion.schemas.schedules import CronSchedule
And you would use this in defining your deployment as per the example given earlier.
For more information and examples regarding schedules, you can visit the Prefect Schedules documentation.Nate
04/17/2024, 2:45 PMNate
04/17/2024, 2:45 PMbuild_from_flow
, agents and infra blocks are deprecated
• always use client
schemas unless you're developing on server-side prefect itselfNate
04/17/2024, 2:46 PMbuild_from_flow
or agents (you should know they're deprecated!), try again with either serve
or deploy
search for both and compile a comprehensive answerMarvin
04/17/2024, 2:47 PMprefect deployment create
command (the deploy
command), or defining the schedule directly in the prefect.yaml
file. Using agents (prefect agent
) or the build_from_flow
method would be the wrong approach as they are either deprecated or not the recommended way in Prefect 2.x.
Here's how you can proceed with the deploy
command and the prefect.yaml
file:
1. Using prefect deployment create
command
- Run prefect deployment create
with the -i
flag to interactively create a deployment and define a CronSchedule when prompted.
2. Defining the schedule in prefect.yaml
file
- You can define a deployment with a schedule directly in your prefect.yaml
file as follows:
yaml
deployments:
- name: my-flow-deployment
flow_location: /path/to/my_flow.py
schedule:
type: cron
cron: '0 0 * * *' # This expression means every day at midnight
timezone: 'America/Chicago'
tags:
- production
- After defining the prefect.yaml
file, you can apply the deployment using prefect deployment apply prefect.yaml
.
Note that the prefect server start
will start the scheduler service that ensures scheduled flow runs are created and executed as per the defined schedules.
For specific details and additional information, refer to the Prefect Schedules documentation.Nate
04/17/2024, 2:47 PMNoah Holm
04/17/2024, 2:48 PMbuild_from_flow
and are aware of its deprecation. Trying to get a grasp of what we need to change. I'm finding it a bit difficult to get a comprehensive view of our options. I have seen this upgrade page.Noah Holm
04/17/2024, 2:51 PMNate
04/17/2024, 2:57 PM.deploy()
is probably what you'll want
then everything else should be the same as before, except where you get your source code. you can either
• use flow.from_source().deploy()
where from_source
does the job of the storage block, ie where to grab source code at runtime
• bake into image, say push=False
on deploy()
, use DeploymentImage
to pass your own Dockerfile
Noah Holm
04/17/2024, 2:59 PM