How would you describe Prefect in a compact way that highlights its features? It's not a workflow manager in the sense that it is not Camunda, it is not Temporal, it is not AWS Step. It's not Conductor by Netflix
05/05/2022, 4:05 PM
It's a general-purpose workflow orchestration framework allowing you to design, build, run, operationalize and observe your data flows at scale. In contrast to other tools you mentioned, it's not tied to any specific infrastructure (such as a specific cloud provider or platform), operates in a hybrid execution model (giving you cloud convenience with on-prem security) and is particularly helpful in governing both state and data dependencies, and giving you visibility when any task in those workflows fails.
Does it answer your question? Is there a specific use case or feature you want to ask about?
05/05/2022, 5:09 PM
I think all the confusion is coming from the "general purpose workflow framework". Most of these tools would qualify themselves as such. And I would not use Prefect instead of Temporal for microservices orchestration. We need to find better words to explain the concepts of data pipeline as code
05/05/2022, 5:13 PM
Perhaps you can replace the word "workflow" with "data flow" to make it unambiguous? All other tools you mentioned have nothing to do with data flows
And don't forget the word "orchestration" :) Camunda is for business process workflows, it doesn't do any orchestration and any data flows
05/05/2022, 5:43 PM
I think data flow is probably the right answer, yes... workflow is inflated 😄