https://prefect.io logo
m

Mike Kovetsky

08/31/2022, 2:16 PM
Hey, Prefect community! I am creating a prefect (v2.0.4) flow for hyperparams search. I use ray[tune]=2.0.0 as a library for tuning. Could you please assist in finding some answers: 1. Does anybody have a working example? I supposed this to be a common task (especially when i saw RayTaskRunner) so I expected to find some working example. 2. I’d like to use my prefect task as a ray trial. So that i can see tasks in the prefect dashboard. Unfortunately, tune.Tuner requires a function as an arg. When i try tune.Tuner(prefect_task), prefect gives the error:
Copy code
RuntimeError: Tasks cannot be run outside of a flow. To call the underlying task function outside of a flow use `task.fn()`.
When i try tune.Tuner(prefect_task.fn), the flow runs successfully. But the task registration is skipped, so I see no progress in the prefect dashboard šŸ˜ž This point is really crucial, because I cannot run parallel tasks with RayTaskRunner, as there are no prefect tasks registered. 3. I wonder what is the best practice to manage max concurrency. Should i specify concurrency-limit 2 on the prefect level? Or maybe on Ray level tune.TuneConfig(max_concurrent_trials=2)? Thank you in advance! šŸ™‚
k

Khuyen Tran

08/31/2022, 3:40 PM
It seems like you try to use the task outside of a flow. Can you give me a sample code of your code for better debugging?
4 Views