Alexander Butler
04/14/2022, 1:57 AM- name: elt-salesforce
flow_location: ./salesforce_flows.py
flow_name: elt-salesforce
tags:
- salesforce
- core
parameters:
destination: "gcp"
schedule:
interval: 3600
Assuming interval is seconds? Can I specify another grain? Can schedule take a dict? If it takes cron, does that take a dict?
Honestly schedule is the primary question point. Everything else is straightforward enough.Kevin Kho
04/14/2022, 2:33 AMAnna Geller
04/14/2022, 9:43 AM- name: crypto_prices_etl_dev
flow_location: /Users/anna/repos/gitops-orion-flows/flows/crypto_prices_etl.py
flow_name: crypto_prices_etl
tags:
- dev
schedule:
interval: 3600
- name: repo_trending_check_prefect_dev
flow_location: ./flows/repo_trending_check.py
flow_name: repo_trending_check
tags:
- dev
parameters:
repo: "prefect"
schedule:
interval: 3600
- name: repo_trending_check_orion_dev
flow_location: /Users/anna/repos/gitops-orion-flows/flows/repo_trending_check.py
flow_name: repo_trending_check
tags:
- dev
parameters:
repo: "prefect"
schedule:
interval: 60
I believe the same definition via Python DeploymentSpec
is much cleaner and easier to understand/change, but YAML is also fine πAlexander Butler
04/14/2022, 2:51 PMschedule
supports cron or different kwargs for interval?Zanie
04/14/2022, 3:12 PMcron: string-here
instead of interval: integer
itβd be loaded as a cron scheduletimedelta fields can be:
timedelta, existing timedelta object
int or float, assumed as seconds
str, following formats work:
[-][DD ][HH:MM]SS[.ffffff]
[Β±]P[DD]DT[HH]H[MM]M[SS]S (ISO 8601 format for timedelta)
terrence
04/14/2022, 3:20 PMAnna Geller
04/14/2022, 4:02 PM