Hi there. I need to schedule a flow to run the first Wednesday of every month. I tried the approach ...

Pedro Machado

11/13/2020, 2:49 AM

Hi there. I need to schedule a flow to run the first Wednesday of every month. I tried the approach outlined here and then ran into the same issue described in the thread. Is it possible to do this with the existing filters? I don't understand the security implications of allowing arbitrary callbacks for functions but it seems like a good feature to have.

Pedro Machado

11/13/2020, 2:50 AM

I was trying to use this logic:

Copy code

def is_first_wednesday(dt):
    first_wednesday = dt.start_of("month").next(pendulum.WEDNESDAY)

    return dt == first_wednesday

clock = clocks.CronClock("0 0 * * 3")
schedule = Schedule(clocks=[clock], filters=[is_first_wednesday])

but I got

Copy code

raise ValidationError("Invalid function reference: {}".format(value))
marshmallow.exceptions.ValidationError: Invalid function reference: <function is_first_wednesday at 0x7fa0e2768ee0>

Chris White

11/13/2020, 2:58 AM

Hi Pedro - all schedules / filters need to be executable in Prefect Cloud, hence the tight restriction that they must come from the pre-defined library. We definitely welcome new filter contributions if they are general / flexible enough!

is_first_weekday_of_month(day_of_week: int)

could be a nice one to PR. In the meantime, because your schedule description only consists of 12 dates that span the year, I recommend using an explicit list + a

DatesClock

as described in this similar issue: https://github.com/PrefectHQ/prefect/issues/3501

Pedro Machado

11/13/2020, 3:40 AM

Thanks Chris. I'll give this a try.

👍 1

3 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.