I've couple of questions would like to know how will perfect addresses these. • Ability to add human workflow. Basically want to inject an approval/reject or some data entry operations by human. Is it possible to achieve that? • Is it possible to use Apache Arrow to pass data between tasks? Apache Arrow seems to be an interesting data reprasentation where by we can avoid extra serialization/deserialization

Hey there @Kesav Kolla, great questions. For the first question, one way to add human workflow to the Prefect flow is by adding a

manual_only

trigger onto your task. This will cause that particular task to enter a

Paused

state when it’s run and can be changed using the UI itself, though this is only compatible with the open-source UI or Prefect Cloud. Additionally, this process will be a bit easier in the next week with the new UI notifications, which will be able to notify you when a task hits a

Paused

state. For your second question, we currently do not have integrations for Arrow in our Task Library, however this would be interesting to explore! If you want, I recommend opening an issue on Github requesting this integration and any configurability that will be the most useful for you.

can this manual_only trigger by added by condition. Basically I wanted this manual intervention based on some results of previous tasks.

We could explore an Arrow

Result

, which is the new serialization interface that was introduced in Prefect 0.11. Triggers are specified on the task object prior to running the flow, and therefore need to be set in advance, but you can write custom triggers if the built-in ones aren’t meeting your needs. A trigger is just a function that accepts upstream states and returns

True

if the task can run

Potentially, you can also use a PAUSE signal within the task to change the task state based on some conditional logic.