Hi! I am new both to Airflow and Prefect I have a scenario that when a file comes into a certain folder on a machine, I will have to run a python program and parse and send the contents of that file to a DB I was able to do it with using directory watchers in python (by using inotify or watchdog), I might have close to 1000 files coming to that directory anytime, so my file watchers would be always running But since discovering about the prefect project, I would want to incorporate prefect in the whole picture I was wondering if there is something of a directory watch trigger (which would trigger a data flow as soon as a file comes into that watch directory) available in prefect? similar to PIN 8

Hi @Sainath, something like that doesn't exist currently, but we're definitely thinking about this use case. The updated thoughts on event-driven flows are in PIN-14 (https://docs.prefect.io/core/PINs/PIN-14-Listener-Flows-2.html). There's a working-group on event driven flows being organized in this slack (cc @Laura Lorenz (she/her)) - if you have feature requests or thoughts on this they'd be more than welcome.

Hey @Sainath, RT to everything @Jim Crist-Harif said but to follow up on the development aspect of PIN 8/PIN 14, if you want to follow along definitely check out the #contributors-event-driven-flows channel in the next few weeks, right now I’m actively collecting use cases like this one 🙂 so I’ve written your use case down in my little collection 😆