https://prefect.io
Join Slack
Is there a way to set the start time of a IntervalClock for a flow using an input parameter?
# ask-community
n
Is there a way to set the start time of a IntervalClock for a flow using an input parameter?
a
What's your use case? Why would you want to do it this way?
n
dockerize flows and trigger them from one parent flow.
a
how is the parametrization of scheduled start_time relevant here? it may sound like a dumb question, but I don't see the connection here
interval schedules are generally meant to be running every X, say every 15 minutes - I would like to understand why parametrizing the start time would matter here given it runs on a regular interval anyway
n
We have to start 150 flows with batches of 10 each and 10mins apart
a
so the problem is batching and limiting concurrency?
n
yes
Copy code
clock_1 = IntervalClock(timedelta(minutes=10),
                        start_date=datetime.now(ist),
                        parameter_defaults={'Frequency':'Start_At_0'})

clock_2 = IntervalClock(timedelta(minutes=10),
                        start_date=datetime.now(ist) + timedelta(minutes=15),
                        parameter_defaults={'Frequency':'Start_At_15'})
can i use something like this?
for the parent flow?
a
Nice, I'm so glad we identified the actual problem here. Using parameters is not a good solution here - instead, it would be better if you set a concurrency limit for this flow - check this page
n
thanks anna, will check
a
to explain why: even if we define the parameters to start the first run at a given time using a parametrized start_date, the interval clock will keep spinning up new flow runs every 10 minutes for each of those clocks, which doesn't solve your actual problem of queueing the "too many" runs - but concurrency limits will help
n
yeah, makes sense!
👍 1
Will try this out Anna 🙂
🙌 1
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by