howdy 🙋 I have a couple of questions regarding best-practices with Prefect. • the first one is regarding a “validation” step (e.g. - if current state in the world already == desired state, then just SKIP the flow entirely). I know i can SKIP a task (by raising skip), but if i wanna skip the entire flow, should i just make sure that the downstream flows depend on it and that’s it? • the second Q is more general about orchestration : let’s say i have an external resource that i want to change to state “GREEN” at some time in the day (every day) and then set back to “BLUE” at some later time in the day (similarly, every day) - in the beginning i thought of doing like a single flow (that would run for example once an hour) and would contain an internal representation the desired schedule - and apply it within the run itself to decide if the state needs to be changed, and if so - to what. Then i figured it might be simple to just run the flow with two different parameters - twice a day (once to make things “Green”, and then again to change them back to “Blue”, with a flow-run parameter — is that even possible in prefect? to have two schedules with different parameters?) --- i’m not sure which one is deemed more “correct”?

#1 What would be the desired state? At a first glance, Terraform seems like a better tool for such use case #2 I'm not sure either

#1 the desired state changes based on the time of day. i.e. on 13:00 UTC we wanna change to green, and then on 22:00 UTC we wanna change to blue — every day, repeatedly. And this change is done via an API call (several actually). Why would Prefect not be suitable for that?

ahh thanks for more info I believe in that case, you should look more into caching - this way you can avoid recomputation for a given time

@Anna Geller the external resource is basically a cloud DB that we need to switch between "high" computation power and "low" computation power based on schedule (in the future we might change it to trigger based on the load on the DB itself). They don't have "auto-scaling" of resources so to avoid over-paying we need to scale it up and down ourselves. Not sure what you mean by recomputation? we don't actually compute anything, we just wanna alter the DB from one configuration to another back and forth (and e.g. keep it with the same "low" config throughout the weekend) at least for the first bullet i think i answered myself and basically if i raise

SKIP

in the validation task, it would cause all downstream tasks to skip too (and the flow itself, ultimately) - which achieves the goal i wanted, I'm just not sure how "correct" it is as far as best-practices go (e.g. would i have been better of with a conditional flow (i.e.

case

) ? the problem with that is that i don't actually have anything to do if the condition is

False

for example, only if it's

True

) for the second part -- i'm still unsure if having two "distinct" run-schedules (one for "green" and one for "blue") for the same flow make sense in an ideal world i would represent the entire schedule of what state should the DB be in at any given moment with like consecutive intervals that are represented in some compact way (and lives inside the flow itself) ---- rather than as just "cron-based schedules for change operations to run in" --- that way the flow could be run arbitrarily often and simply "adjust" the DB to the correct state when time fits the desired "mapping" (or, when other unique conditions apply -- perhaps) - if the DB doesn't happen to be in the desired state already kind of like polling behavior

I see - this is related https://discourse.prefect.io/t/pausing-flow-group-schedule-with-graphql/359/5

hmm, not sure why is it similar? i don’t need to pause anything 🤷‍♂️

similar because it's also about DBA maintenance

@Anna Geller the fact that the resource happens to be a DB doesn't really have any bearing on the task (which is why I intentionally just called it an "external resource" at first and only elaborated later) - it could have been a "smart air-conditioner" that i'm trying to control the temperature for on schedule --- Prefect is just a convenient tool for running a bunch of tasks with scheduling, failure-handling, visibility, alerting, etc. the issue i'm asking about has to do with scheduling flows (and what's the best way to approach it), not with databases 🙂 also, it's a solved problem (my current solution work) - i'm more just consulting on what might be considered best-practice -- in the eyes of the orchestration experts and regarding polling - i think my use of the word might have been a bit incorrect --- i'm not concerned (yet) with reacting to an external state (i will in the future though) when i said "polling" i meant more like a high-frequency rule-enforcer rather than something that reacts to actual events in the real-world in other words, rather than Prefect flow running at hour A and hour B everyday based on two schedules (each with their own params "X" and "Y"), i could have a single flow running EVERY hour - and applying internally the logic that determines what should be the desired parameters at the current run - based on internally stored rules (e.g. if it's between 4AM and 5AM and today is a tuesday, do "X", otherwise, if it's between 9AM and 10M do "Y" and otherwise do nothing and leave things as they are") anyway, like i said, it's just kind of thinking out loud what would be "best" , it's not an actual unsolved problem - so not urgent either...

I don't have any best practice recommendations here, Prefect can cover a wide range of scheduling and you can e.g. even configure a custom RRULE schedule if things like interval or CRON don't work well for you glad this community helps you think out-loud and rubber duck 🙌 :marvin-duck: