@Kevin Kho hi kevin, is there any plan to support Celery Runner? at most map scenario, we just want a distribute queue to run many task and a rich web ui to manage these task in a batch. so sometime we don't need a complicated engine like dask to do this, because some time it will make some confused problem( e.g., run a task twice )
k
Kevin Kho
06/29/2022, 4:18 PM
Not soon because the Celery task queue model is different from a batch orchestrator so it needs more thought
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.