Hello everyone, I hope you are well. We have an AWS Kinesis Data Analytics (KDA) application. A task in one of my flows is responsible for starting the KDA via the Boto3 library (start_application). My next task will start when the KDA stops, so I need to know the app status, so there is another API called “describe_application” to see its status. If the app status is “READY”, then I can continue with the next task. What is the best way to design that flow? 1. Using "waiter" to see the status, but I could not find related waiters for KDAs. There are good examples for Amazon Batch. 2. Creating a separate task to call the “describe_application” in a scheduled way. As far as I understand schedules are generally used for flows. Could you please advise me on this?

Hi @Ismail Cenik! Can you just use the

describe_application

every minute or so in Python code, and then just return when the app status is “READY”? Wrap this in a task, and then set this task as an upstream dependency for your other tasks.

Hi, thank you for the answer Around after 5 mins

I would make a recursive function to check every 5 mins then return when it’s “READY”

Ok, thanks. Let me try this approach.

Thanks @Kevin Kho instead of polling for 'READY', is there a way to have KDA register a callback function and let prefect knows? Or can prefect monitor a remote file (on S3) and take action upon arrival? @Ismail Cenik

Hi @Eric! You can’t register a callback function. The approach here would be to use the S3 Put Action to trigger a Lambda to start a Prefect Flow

can we set up a file watcher?

At the moment Prefect doesn’t have such a capability but it is something we do plan to have in the future.