Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Orion feature request/idea: now that the tool is ‘free from the DAG’ I could image that supporting new ‘execution’ layers besides Dask will be much easier. Has the team thought about added native support for <http://Ray.io|Ray.io> clusters?

Hey Bob, I've prototyped such an executor and it's definitely very easy to support. Keep an eye out for future announcements in this vein :slightly_smiling_face:

Regarding Prefect as an abstraction on top of Dask and Ray. It would be very powerful IMHO if Prefect was able to promote some existing dask layer election parameters as first-class Prefect APIs. I’m thinking of  `cluster_kwargs` like `processes`, `n_workers`, and `threads_per_worker` that let you tune the flow execution. I think this is a killer Dask feature and would love the same high-level API available when running on Ray. No doubt the implementation would be non-trivial.