https://prefect.io logo
Title
c

Christian Vogel

07/07/2022, 10:59 AM
Hi Prefect Community, I am currently receiving an error when using a local (on my pc) prefect agent which pulls from a work queue in the the prefect cloud. The error only occurs when I try to use the RayTaskRunner:
(begin_task_run pid=141324) ImportError: cannot import name 'SubprocessFlowRunner' from partially initialized module 'prefect.flow_runners' (most likely due to a circular import) (/home/christian/Documents/ray_and_prefect/env/lib/python3.9/site-packages/prefect/flow_runners/__init__.py)
I am using the following dependencies: prefect==2.0b7 prefect-ray==0.1.0 ray==1.13.0 Apparently I am doing something wrong with my dependencies or when I am importing them. Do you have any idea?
1
This is my flow and deployment:
from prefect import task, flow
from prefect_ray import RayTaskRunner
from prefect.deployments import DeploymentSpec


@task
def say_hello(name):
    print(f"hello {name}")


@flow(name="temp-flow-example", task_runner=RayTaskRunner(address="auto"))
def greetings():
    say_hello("Ford")


DeploymentSpec(
    name="temp-flow-example",
    flow=greetings
)
What confuses me a bit, is the fact, that the flow only fails when pulled from the cloud and seems to work when i run it directly locally via python and a main method.
Could it be related to the fact that I am running my agent in a virtual environment? Probably in the background the following is executed:
from prefect.flow_runners import SubprocessFlowRunner
which them leads to some import issues because of the already imported:
from prefect_ray import RayTaskRunner
Is that possible?
a

Anna Geller

07/07/2022, 11:57 AM
do you run your agent in the same virtual environment as you run your local process? perhaps you can explicitly define your flow runner with the virtual environment you use for your agent?
flow_runner=SubprocessFlowRunner(condaenv="yourcondavenv"),
I was trying to replicate and sth also didn't work for me, even though I got a different error message using the same versions as you did
my error was:
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-07_14-08-35_008202_6668/logs/dashboard.log'
wow it took me forever to set this up but this helped https://docs.ray.io/en/master/ray-overview/installation.html#m1-mac-apple-silicon-support looks like you need to be super careful about your environment with Ray and grpcio package - do you use Conda? on which machine do you run it?
c

Christian Vogel

07/07/2022, 12:24 PM
Tried specifying the SubprocessFlowRunner with the virtual env path in my case. but did not help
I am running a venv on a linux ubuntu machine
a

Anna Geller

07/07/2022, 12:25 PM
yup you're right, I was able to reproduce your error, thanks for reporting that @Marvin open "Flow run with Ray and agent throws: `ImportError: cannot import name 'SubprocessFlowRunner' from partially initialized module 'prefect.flow_runners'`"
c

Christian Vogel

07/07/2022, 12:48 PM
thanks for the quick support as always
🙌 1