Hi people, I was wondering what people's experienc...
# ask-community
m
Hi people, I was wondering what people's experience with Ray + Prefect is like and whether you use it in conjunction with Prefect through the Ray TaskRunner, submit a RayJob instead or just use it a standalone framework alongside Prefect? Our team primary works with large distributed inferencing and data processing workflows.
Some background, we are modernizing our infra stack and making optimization, partly by finally taking the plunge of updating from prefect 2 -> prefect 3. Ray seems to offer lot's of improvements and quality of life improvements to data loading, resources scheduling and all the ML niceties that come with it. One the piqued our interest was using it in Prefect through the task runner allowing for nice mapping and join operations in a distributed manner, as well as the ability to use partial gpu resources!!! My team and I are really interested in hearing how the community uses Prefect with Ray!
r
We’ve mostly used Ray alongside Prefect rather than tightly integrating it. Prefect handles the workflow orchestration and scheduling, while Ray takes care of the distributed compute parts. For larger inference and data processing workloads that setup has worked pretty well so far. I haven’t personally used the Ray TaskRunner yet, but I’ve seen a few teams mention it when they want a tighter integration.
cool cool 2
m
Ohhhh thanks Reynalyn, so you submit a RayJob or using ray.remote from the Prefect job. We are evaluating options right now because we have found Prefect's state tracking to be notoriously unreliable at scale.
c
@Reynalyn Payumo did you have any issues or learning you could share regarding dependency management? We're finding that the ray head node needs to have a superset of the dependencies that ray workers need (due to pickling) and this is causing versioning conflicts. We're currently exploring alternative solutions.