https://prefect.io logo
Title
d

Diego Piloni

06/17/2021, 11:46 AM
Hi 👋 ! First of all, thank you for such a great tool, I really enjoy using prefect 🙂 I'm using prefect-ui with hourly schedule to run an ETL flow. The agent is in a raspberrypi4 🙈 , and it sometimes get discconected from the network (Maybe I should stop using a Raspberry for this, I know). When this happen I see that all the missing hourly flows start executing when the agent is turned on (all at the same time). Is there any way to configure prefect to not start executing flows after the schedule time? Should I do it programatticaly? Or maybe just don't allow it to execute the same flow at the same time. Here is an example where many flows started running together:
🙏 1
k

Kevin Kho

06/17/2021, 1:26 PM
Hi @Diego Piloni, in Standard tier and up we offer Flow automations where you can say “don’t run a flow if it’s late by X amount”. What are you using the raspberry pi for? I’m curious
d

Diego Piloni

06/17/2021, 4:24 PM
Nice to know ! I'm interested in this feature so I'll check the standard tier for sure. About the raspberry: Basically I'm new at a startup (in Argentina) and I still havent had time to learn how to deploy in the company's cloud instances, so since my ETLs use the compute power of a Data Warehouse I don't need too much compute in the agent host, so I decided to use a raspberry that was at hand to run the schedules. 😛 The problem is when the raspberry get's discconected from the internet, ISP's are not so stable here. Thanks for your help, @Kevin Kho!
k

Kevin Kho

06/17/2021, 4:25 PM
Oh haha! That’s cool!
😅 1
d

Diego Piloni

06/17/2021, 4:33 PM
About executing the same flow twice concurrently at the same time, is there some way to avoid this?
k

Kevin Kho

06/17/2021, 4:41 PM
Is it because the schedule got delayed? Or is there multiple schedules/run creations and you just want something to block 2 of the same flow from running concurrently?
d

Diego Piloni

06/17/2021, 4:45 PM
In any case where there are mulitple executions of the same flow I would like to be able to block all except the one who started first.
k

Kevin Kho

06/17/2021, 4:46 PM
I’m starting to sound like I’m selling aggressively, but limiting flow run concurrency is also on the standard tier 😅
d

Diego Piloni

06/17/2021, 4:47 PM
hahaha, thats ok ! it's nice to know it again, I will better go and check the tier features 😛 Thanks a lot for your time @Kevin Kho, nice to meet you 🙂
k

Kevin Kho

06/17/2021, 4:47 PM
Nice to meet you also!