https://prefect.io
Join Slack
Hi team, we are facing an odd issue with scheduling flows. We have a python script that schedules fl...
# prefect-server
z
Hi team, we are facing an odd issue with scheduling flows. We have a python script that schedules flows. When the script gets run, it duplicates the schedule for one particular flow, for the next day. For example, if there is a flow scheduled for 1 am tomorrow, when we re-run the build_flows script, it will create another flow for that same time the next day, but not any subsequent days
this is the code snippet for the scheduler
Copy code
if flow.name == 'ppi_parent_flow':
            clocks = []
            for prop_source, prop_source_config in US_PROPERTIES_CONFIG_MAP.items():
                clock = CronClock(cron=prop_source_config['schedule'], parameter_defaults={
                    PREFECT_DATASET_PARAM_NAME: prop_source, PREFECT_RUN_ENVIRONMENT_PARAM_NAME: run_env})
                clocks.append(clock)
            schedule = Schedule(clocks=clocks)
            flow.schedule = schedule
this is an example config with all the schedules
Copy code
US_PROPERTIES_CONFIG_MAP = {
    'DATASET': {
        'schedule': '0 5 * * 1-5'
    }
}
any help is appreciated, thanks!
k
Do you have a schedule that was configured through the UI?
z
no it should all be through the API
k
I can’t immediately see anything wrong
z
@Kevin Kho for the clock schedule, is there a parameter to make it overwrite existing schedules, or something like that?
k
flow registration will always overwrite the existing schedule
6 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by