Hi, I have a question about Prefect working with Ray https://prefecthq.github.io/prefect-ray/ It is possible to run task on many machines using this setup? Is it creating a Ray Cluster then?

I think you have to set up your own ray cluster and specify that it should connect to it if you want something that runs on multiple machines!

Hi Piotr, Oscar is correct. By default, the Ray task runner will start up a local Ray cluster. If you want to use Ray to distribute Prefect tasks across multiple machines, you'll need to start a Ray cluster and then use the

address

parameter when creating the task runner, like so:

from prefect import flow
from prefect_ray.task_runners import RayTaskRunner

@flow(task_runner=RayTaskRunner(address="ray://<your-ray-cluster-ip>:8786"))
def my_flow():
    ...