Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
i
Ilhom Hayot o'g'li
05/23/2022, 12:08 PMHI! I wanted to run 2 or more flows independently for ETL processes with cron. What are the best practices? Is there some way of doing so ?
a
Anna Geller
05/23/2022, 12:34 PMHi @Ilhom Hayot o'g'li, you can do that using Prefect. Specifically for CRON check this page https://orion-docs.prefect.io/concepts/schedules/#cronschedule
from prefect.deployments import DeploymentSpec
from prefect.orion.schemas.schedules import CronSchedule
DeploymentSpec(
name="cron-schedule-deployment",
flow_location="/path/to/flow.py",
schedule=CronSchedule(
cron="0 0 * * *",
timezone="America/New_York"),
)i
Ilhom Hayot o'g'li
05/23/2022, 12:42 PMok I got about cron part...
How about running 2 flows independently?
a
Anna Geller
05/23/2022, 12:44 PMthat works natively with Prefect, no need to do anything else here 🙂 you can just schedule two flows with the same schedule/cron expression
i
Ilhom Hayot o'g'li
05/23/2022, 1:10 PMok lets explain with img its faster.
I need flow that runs other flows
a
Anna Geller
05/23/2022, 1:55 PMthe syntax I shared before was for Prefect 2.0 - in Prefect 2.0 there is a concept of subflows allowing you to call flows from other flows by calling those as any other function
here is the difference between 1.0 and 2.0 https://discourse.prefect.io/t/should-i-start-with-prefect-2-0-orion-skipping-prefect-1-0/544
here is about subflows: https://orion-docs.prefect.io/concepts/flows/#subflows
i
Ilhom Hayot o'g'li
05/23/2022, 1:57 PMwhat is about prefect==1.2.0 is that possible in that version too?
a
Anna Geller
05/23/2022, 2:01 PMno, Prefect 2.0 is a completely new product with a different API behind it
in Prefect 1.0 (we're talking 1.0 as a generation of a product, not a specific version) there is a concept of flow of flows (aka "orchestrator pattern") allowing you to call a flow from another flow - you can read more about it here (including many examples)
i
Ilhom Hayot o'g'li
05/23/2022, 2:38 PMThis was exact answer which I found from your link
Thank you for your care👍
a
Anna Geller
05/23/2022, 2:40 PMthat's fantastic, so glad this has helped 🙌
please only cross-check the syntax - the article was using a much older Prefect version
i
Ilhom Hayot o'g'li
05/23/2022, 2:42 PMsure thanks
👍 1