Zhibin Dai

    Zhibin Dai

    3 months ago
    Hi team, we are facing an odd issue with scheduling flows. We have a python script that schedules flows. When the script gets run, it duplicates the schedule for one particular flow, for the next day. For example, if there is a flow scheduled for 1 am tomorrow, when we re-run the build_flows script, it will create another flow for that same time the next day, but not any subsequent days
    this is the code snippet for the scheduler
    if flow.name == 'ppi_parent_flow':
                clocks = []
                for prop_source, prop_source_config in US_PROPERTIES_CONFIG_MAP.items():
                    clock = CronClock(cron=prop_source_config['schedule'], parameter_defaults={
                        PREFECT_DATASET_PARAM_NAME: prop_source, PREFECT_RUN_ENVIRONMENT_PARAM_NAME: run_env})
                    clocks.append(clock)
                schedule = Schedule(clocks=clocks)
                flow.schedule = schedule
    this is an example config with all the schedules
    US_PROPERTIES_CONFIG_MAP = {
        'DATASET': {
            'schedule': '0 5 * * 1-5'
        }
    }
    any help is appreciated, thanks!
    Kevin Kho

    Kevin Kho

    3 months ago
    Do you have a schedule that was configured through the UI?
    Zhibin Dai

    Zhibin Dai

    3 months ago
    no it should all be through the API
    Kevin Kho

    Kevin Kho

    3 months ago
    I can’t immediately see anything wrong
    Zhibin Dai

    Zhibin Dai

    3 months ago
    @Kevin Kho for the clock schedule, is there a parameter to make it overwrite existing schedules, or something like that?
    Kevin Kho

    Kevin Kho

    3 months ago
    flow registration will always overwrite the existing schedule