Silly question, where does Prefect fit in with Python Celery and Message Queues / Brokers (e.g. RabbitMQ)? It seems like some of the tasks, such as restarting (or retrying) a long running asynchronous job can be implemented in either of the above solutions. I was wondering what would be the difference and the tradeoffs to using each of the above solutions.

I don't think it's a silly question. I'm evaluating Prefect as a Celery replacement. It seems so far that Prefect does what Celery does, but it has better observability, better state tracking, easier complex job creation (multiple tasks chained in Celery vs Flows in Prefect), and very nice UI (which requires (celery-flower for Celery). Seems like a really nice upgrade to Celery so far to me.

Thanks @Shawn Holyoak I appreciate the honest response!

We’re actively investigating ways to be a better drop-in replacement for Celery 😉 I bet our product team would be interested in chatting with ya’ll about your needs.

Thanks @Zanie, I’d love that 🙂

Plus 1 for me

@Ofir and @Shawn Holyoak please DM me your email address and I'll find some time for us to chat. As @Zanie said, we're actively exploring how Prefect could better solve the same problems that people currently use Celery for.