Serdar Tumgoren
04/04/2024, 4:01 PMprefect deploy --all
. I noticed that whenever it runs, it clobbers the schedules that I've created manually for various deployments using the Prefect Cloud UI. I see in the docs that you can set a cron schedule for all deployments using prefect deploy
or in prefect.yaml
, but afaict that only covers the scenario of a single schedule for all deployments. Is there way I can update the below snippet from my prefect.yaml
so that there's a separate schedule for each entry under deployments? Is it as simple as adding the schedule
section to each deployment config, as opposed to setting it at as a default at the deployments
level, per these docs?
deployments:
- name: metadata
entrypoint: flows/civic_plus/metadata.py:collect
work_pool: *default_work_pool
description: "Scrapes agencies for metadata about agenda documents."
parameters:
start_date: ''
end_date: ''
override_urls: []
- name: assets
entrypoint: flows/civic_plus/assets.py:upload
work_pool: *default_work_pool
description: "Uploads agenda documents to Google Cloud Storage."
- name: document-cloud-submit
entrypoint: flows/civic_plus/submit.py:submit
work_pool: *default_work_pool
description: "Submits documents to DocumentCloud."
- name: document-cloud-extract-text
entrypoint: flows/civic_plus/extract_text.py:extract_text
work_pool: *default_work_pool
description: "Loads the extracted document text into the Agenda Watch database."
Serdar Tumgoren
04/04/2024, 4:46 PMprefect.yaml
using a yaml anchor and alias, per this example. I'll give that a go but welcome alternate suggestions or corrections if that's not the right path for my situation...Tess Dicker
04/04/2024, 4:51 PMSerdar Tumgoren
04/05/2024, 9:15 PMprefect.yaml
to include the schedules
section for each deployment step, as below. Worked like a charm!
deployments:
- name: metadata
entrypoint: flows/civic_plus/metadata.py:collect
schedules:
- cron:"0 */12 * * *"