Say I have a flow that processes data and publishes a manual

Question

Say I have a flow that processes data and publishes a manual computer vision annotation job to an annotation server how might I schedule a flow to poll the annotation job to publish some information once the job has been completed It may take hours days or maybe a week for this to finish so polling every 4 hours or something may make sense Would this be an application for `pause flow run` using the `reschedule` argument Or is there another way I can create a `cron` like schedule for a parameterized flow to run until it successfully completes

Geoffrey Keating · Answer

Alternatively I could have a flow that tracks down the jobs datasets that need to be published but unfortunately that is a little bit harder that scheduling a polling operation

Jake Kaplan · Answer

If i m understanding right would it work for your use case to kick off another flow run to publish the information once your original flow run completes This way you wouldn t need to poll at all

Jake Kaplan · Answer

theres a few ways to do this You can call `run deployment ` inside of your flow run maybe inside of the `on completion` here is a for how to use run deployment Additionally if you re using Prefect Cloud use can use where you can trigger a flow to kick off after success of another flow

Geoffrey Keating · Answer

That s correct but I don t really know when the annotation job will be ready to pull down So something like `run deployment` maybe I could set retries to a large number and poll the status until it is completed that way

Jake Kaplan · Answer

Ah got it so theres kind of a hidden middle step flow1 completes annotation job not a flow happens in some indeterminate amount of time flow2 needs to run

Geoffrey Keating · Answer

Right think of some middle blob of time that is on the scale of human work days week and wrangling a web hook out of the annotation engine would be quite a bit of effort