Is Prefect applicable for stateful transactional workflows?
07/21/2023, 7:09 PM
It's python - so if you can do the work for your 'stateful transactional workflow' in Python - you MIGHT be able to do it with prefect.
07/21/2023, 7:17 PM
hi @Ofir prefect certainly helps with the stateful part of workflows, I'd be curious what you have in mind as far as the transactional part goes
07/22/2023, 11:20 AM
@Peter@Nate thanks for the reply! I’ll provide an example.
A simple example of Amazon, e.g. an E-commerce scenario could be an order placement operation, which could consist of several stages:
1. Validate order
2. Update inventory
3. Deduct payment
If any of these steps fails, the operation should be undone right?What happens if the payment in step (3) was unsuccessful?
The item that was pulled out from the inventory needs to be put back in or in other words, we need to rollback / we need a compensating transaction.
Is Prefect suited for these types of workflows (which Zeebe handles) or is Prefect primarily for stateless DAGs for Data Science / Data Engineering tasks?
An order from Amazon goes through a state machine:
pending -> placed -> paid -> shipped -> etc.