@Marvin I’m looking at the ray task runner. do you know if we can leverage ray’s capacity for pushing code with it?

It seems there is no direct mention of Ray's capacity for pushing code as understood in Prefect's context directly in the docs. However, Prefect provides a general mechanism for handling code deployment and push actions to different environments using cloud storage solutions such as S3. This mechanism is detailed in Prefect's deployment orchestration capabilities. For Ray-specific parallel task execution, you can use

RayTaskRunner

as explained here: - Install

prefect-ray

with

pip install -U prefect-ray

. - Import

RayTaskRunner

from

prefect_ray.task_runners

. - Specify

task_runner=RayTaskRunner

in your flow definition. If your requirement is specific to deploying Prefect flows that utilize Ray and push the code to remote storage, such customization can be achieved using the push and pull sections in Prefect's deployment steps. For more details on code push actions, refer to: * https://docs.prefect.io/latest/guides/prefect-deploy/ * https://docs.prefect.io/latest/guides/dask-ray-task-runners/ Does this help, or do you need more specific information on something else?

@Marvin normally we can pass vars like

runtime_env={
  “working_dir”: …

to ray.init — can we do that?

Yes, you can pass

runtime_env

and other initialization arguments to

ray.init

when using

RayTaskRunner

. The

RayTaskRunner

accepts an optional

init_kwargs

parameter where you can specify these configurations. Here's an example:

python
from prefect import flow
from prefect_ray.task_runners import RayTaskRunner

@flow(task_runner=RayTaskRunner(init_kwargs={"runtime_env": {"working_dir": "/path/to/your/dir"}}))
def my_flow():
    # Your flow logic here
    ...

# Running the flow
my_flow()

The

init_kwargs

allows you to pass any arguments that

ray.init

accepts, including

runtime_env

. For more information, you can refer to the Prefect documentation on task runners: * https://docs.prefect.io/latest/concepts/task-runners/ * https://docs.prefect.io/latest/guides/dask-ray-task-runners/ Does this solve your query, or is there anything else you need?

hi @Gabriel Lespérance - did marvin answer your question here? i don’t directly know the answer to your question but would be interested in expanding the ray task runner to allow this if we don’t currently

@Nate yeah it did — I have good experience with ray so alhtough I havent tested it makes sense to me

👍