Silly question, where does Prefect fit in with Pyt...
# ask-community
o
Silly question, where does Prefect fit in with Python Celery and Message Queues / Brokers (e.g. RabbitMQ)? It seems like some of the tasks, such as restarting (or retrying) a long running asynchronous job can be implemented in either of the above solutions. I was wondering what would be the difference and the tradeoffs to using each of the above solutions.
s
I don't think it's a silly question. I'm evaluating Prefect as a Celery replacement. It seems so far that Prefect does what Celery does, but it has better observability, better state tracking, easier complex job creation (multiple tasks chained in Celery vs Flows in Prefect), and very nice UI (which requires (celery-flower for Celery). Seems like a really nice upgrade to Celery so far to me.
o
Thanks @Shawn Holyoak I appreciate the honest response!
👍 1
Under the hood, both Prefect and Celery have persistency in some shape or form (e.g. PostgreSQL) and message queues in their underlying implementation?
z
We’re actively investigating ways to be a better drop-in replacement for Celery 😉 I bet our product team would be interested in chatting with ya’ll about your needs.
upvote 2
o
Thanks @Zanie, I’d love that 🙂
s
Plus 1 for me
j
@Ofir and @Shawn Holyoak please DM me your email address and I'll find some time for us to chat. As @Zanie said, we're actively exploring how Prefect could better solve the same problems that people currently use Celery for.
🙌 4