This message was deleted.

@Marvin hi

Hello! How can I assist you today?

@Marvin when i deploy a flow, if my worker is suddenly turn on, all the scheduled run are run. How do I make the schedule not catch up but only run the next one scheduled?

In order to assist you effectively, could you please confirm which version of Prefect you are currently using? The methods to handle scheduling behavior can differ between Prefect 1.x, 2.x, and 3.x.

@Marvin It's Prefect 3.0. Here's my deployment in prefect.yaml

- name: intraday_eod_auto

tags: - stock price pipeline description: Automatically fetch intraday stock price into clickhouse DB entrypoint: src/api/flow_deployment.py:intraday_eod parameters: { check_first_time_migration: false } work_pool: name: local-wp work_queue_name: primary-queue job_variables: {} concurrency_limit: 1 collision_strategy: CANCEL_NEW enforce_parameter_schema: true schedules: - rrule: RRULE:FREQ=MINUTELY;INTERVAL=1;BYDAY=MO,TU,WE,TH,FR;BYHOUR=9,10,11,13,14,15 timezone: Asia/Bangkok active: true max_active_runs: 1 catchup: false version:

In Prefect 3.0, you can control whether scheduled runs should 'catch up' if they were missed during downtime by using the

catchup

parameter in your deployment schedule. Setting

catchup: false

should prevent missed runs from being executed when your worker resumes. This behavior is already configured in your

prefect.yaml

. Unfortunately, there seems to be an issue or misunderstanding regarding the enforcement of

catchup: false

. You might want to check for any updates or recent patches that could address this behavior. Additionally, ensure that your Prefect instance is running the correct configuration by restarting your server and workers. For more details on scheduling in Prefect 3.0, you can refer to their scheduling documentation. If this persists, you can explore related GitHub issues discussed in this thread to see if there are any ongoing improvements or community suggestions regarding catch-up behavior.

@Marvin I can not see anything related to catchup function. Is there any way when I start prefect server and prefect worker, all the late run to be cancel, and that worker only execute the run that is scheduled next

To ensure that only the next scheduled run is executed and all late runs are canceled when you start your Prefect server and worker, you can use the

collision_strategy

parameter with the value

CANCEL

. Here's how you can configure it: 1. Set

collision_strategy

to

CANCEL

: This ensures that if there are multiple pending runs, only the most recently scheduled one is kept. 2. Ensure

catchup

is set to

false

: This prevents older runs from being added when the worker starts. Your

prefect.yaml

configuration should look something like this:

- name: intraday_eod_auto
  tags:
  - stock price pipeline
  description: Automatically fetch intraday stock price into clickhouse DB
  entrypoint: src/api/flow_deployment.py:intraday_eod
  parameters: {
    check_first_time_migration: false
  }
  work_pool:
    name: local-wp
    work_queue_name: primary-queue
    job_variables: {}
  concurrency_limit: 1
  collision_strategy: CANCEL
  enforce_parameter_schema: true
  schedules:
  - rrule: RRULE:FREQ=MINUTELY;INTERVAL=1;BYDAY=MO,TU,WE,TH,FR;BYHOUR=9,10,11,13,14,15
    timezone: Asia/Bangkok
    active: true
    max_active_runs: 1
    catchup: false
  version:

These settings should help you achieve the behavior where only the next upcoming run is kept, and any runs that would have been executed during downtime are ignored. For further reading, you can explore more in the scheduling guide and related GitHub issues for community discussions and potential updates.