Hi all. I'm new to Prefect (and development). I had a doubt. I understand that one can use prefect to set up flows that run tasks in a particular order among other things. Does it also implement queueing? So far in my projects I've only used redis-celery to run big tasks in a queue. PS, I've mostly worked in web dev.

Hi @Sumant Agnihotri, welcome! Prefect is not a queueing framework, in that it doesn’t offer you fine-grained control over its queueing mechanisms. It attempts to run work as close to its scheduled time as possible. However, work is queued in the sense that if you don’t have workers or resources available to handle a scheduled job, it will not be lost; once resources become available, your agents can pick it up.

@Jeremiah Thanks a lot for the detailed reply. Yea, I don't need direct control of the queue. I have a web app with flow as follows: downloading data from user, running it through a bunch of machine learning models and then generating graphs and stuff. Problem I was facing right now is that if more than 5 instances of this flow run concurrently, it crashes the server. I think prefect can help me out here.

Hi Sumant, yes, I think Prefect can help. In the near future, we’ll be adding more features around limiting flow concurrency through the API, which would help you even further in this instance (you could, for example, only allow 1 flow to run at a time)