https://prefect.io
Join Slack
HI! I wanted to run 2 or more flows independently for ETL processes with cron. What are the best pra...
# ask-community
i
HI! I wanted to run 2 or more flows independently for ETL processes with cron. What are the best practices? Is there some way of doing so ?
a
Hi @Ilhom Hayot o'g'li, you can do that using Prefect. Specifically for CRON check this page https://orion-docs.prefect.io/concepts/schedules/#cronschedule
Copy code
from prefect.deployments import DeploymentSpec
from prefect.orion.schemas.schedules import CronSchedule

DeploymentSpec(
    name="cron-schedule-deployment",
    flow_location="/path/to/flow.py",
    schedule=CronSchedule(
        cron="0 0 * * *",
        timezone="America/New_York"),
)
i
ok I got about cron part... How about running 2 flows independently?
a
that works natively with Prefect, no need to do anything else here 🙂 you can just schedule two flows with the same schedule/cron expression
i
ok lets explain with img its faster. I need flow that runs other flows
a
the syntax I shared before was for Prefect 2.0 - in Prefect 2.0 there is a concept of subflows allowing you to call flows from other flows by calling those as any other function
i
what is about prefect==1.2.0 is that possible in that version too?
a
no, Prefect 2.0 is a completely new product with a different API behind it
in Prefect 1.0 (we're talking 1.0 as a generation of a product, not a specific version) there is a concept of flow of flows (aka "orchestrator pattern") allowing you to call a flow from another flow - you can read more about it here (including many examples)
i
This was exact answer which I found from your link
Thank you for your care👍
a
that's fantastic, so glad this has helped 🙌
please only cross-check the syntax - the article was using a much older Prefect version
i
sure thanks
👍 1
5 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by