07/21/2023, 6:44 PM
Saga (1/2) - do you implement the Saga pattern in Prefect for failover/rollback/compensation? One thing that bothered me for a while is what happens when a Prefect Deployment/Workflow interacts with multiple microservices, e.g. via REST API, to orchestrate a complex data pipeline. Each interaction maps to a CRUD operation, so you may have: 1. First task - updates a db (that belongs to microservice A) via REST API to microservice A 2. Second task - updates another db (that belongs to microservice B) via RES TAPI to microservice B Now how do you implement failover or rollback a change if the Second task failed? Would you undo the first task? can you always undo it? Or perhaps Prefect is not suited for cross-db transactions stateful workloads and I should use Zeebe or similar? Thanks!