Thomas La Piana

10/07/2020, 4:22 PM
does prefect have the "sensor" concept from airflow? triggering a flow based on a change in an external object's state?


10/07/2020, 4:27 PM
Hey Thomas - Airflow sensors are just tasks that ping until a condition is met, at which time they complete and allow downstream tasks to run. Therefore, this is a very easy pattern to implement in Prefect, just by writing whatever Python logic you need to check for the condition. If you actually want to trigger a flow based on an external object, then you’re talking about something Airflow actually can not do (sensors only exist within a single DAG) - but Prefect can. You could either use an external service to run a flow via API (not needing a sensor at all) - here’s an example using AWS Lambda. Alternatively you could combine a sensor with a FlowRunTask to kick off a new flow from the flow that contains the sensor.

Thomas La Piana

10/08/2020, 3:27 AM
thank you @Jeremiah! huge help
👍 1