Hello everyone. We primarily use prefect for ETL j...
# ask-community
k
Hello everyone. We primarily use prefect for ETL jobs, loading data from one DB/file to our own DB. We use a master flow (flow of flows) which launches the sub-flows based on their dependency and scheduled_start_time. Sometimes, the systems we get our data from, report a delay by x hours. Is there any way (preferably through the UI) to pause this one flow and restart it later manually? Since there are more flows in the master flow, the other should continue as planned.
k
Hey @Kamil Gorszczyk, would that pause be manual or you want to do it through code?
k
Preferably manual through the Web UI. Through code I could do it by raising the PAUSE signal but the mail we receive to hold our flow is just to unstructured to check it via python. I already thought about going through the Exchange Inbox and checking the last received mails but also doing natural language processing is something I wanted to avoid. And unfortunately there is no way I could convince them to send me a trigger or at least use the same wording in the mail.
k
Seems like there is no option, let me confirm with the team
Hey, so if you PAUSE from the API, it doesn’t propagate to the task because the task is not always checking for the state update. So the only way to achieve this is to raise through code unfortunately. Some people have a lot of retries with a big delay and just let the task error if the upstream process is not done yet.