https://prefect.io
Join Slack
Hi Prefect Community, I am currently receiving an error when using a local (on my pc) prefect agent ...
# ask-community
c
Hi Prefect Community, I am currently receiving an error when using a local (on my pc) prefect agent which pulls from a work queue in the the prefect cloud. The error only occurs when I try to use the RayTaskRunner:
Copy code
(begin_task_run pid=141324) ImportError: cannot import name 'SubprocessFlowRunner' from partially initialized module 'prefect.flow_runners' (most likely due to a circular import) (/home/christian/Documents/ray_and_prefect/env/lib/python3.9/site-packages/prefect/flow_runners/__init__.py)
I am using the following dependencies: prefect==2.0b7 prefect-ray==0.1.0 ray==1.13.0 Apparently I am doing something wrong with my dependencies or when I am importing them. Do you have any idea?
✅ 1
This is my flow and deployment:
Copy code
from prefect import task, flow
from prefect_ray import RayTaskRunner
from prefect.deployments import DeploymentSpec


@task
def say_hello(name):
    print(f"hello {name}")


@flow(name="temp-flow-example", task_runner=RayTaskRunner(address="auto"))
def greetings():
    say_hello("Ford")


DeploymentSpec(
    name="temp-flow-example",
    flow=greetings
)
What confuses me a bit, is the fact, that the flow only fails when pulled from the cloud and seems to work when i run it directly locally via python and a main method.
Could it be related to the fact that I am running my agent in a virtual environment? Probably in the background the following is executed:
Copy code
from prefect.flow_runners import SubprocessFlowRunner
which them leads to some import issues because of the already imported:
Copy code
from prefect_ray import RayTaskRunner
Is that possible?
a
do you run your agent in the same virtual environment as you run your local process? perhaps you can explicitly define your flow runner with the virtual environment you use for your agent?
Copy code
flow_runner=SubprocessFlowRunner(condaenv="yourcondavenv"),
I was trying to replicate and sth also didn't work for me, even though I got a different error message using the same versions as you did
my error was:
Copy code
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-07_14-08-35_008202_6668/logs/dashboard.log'
wow it took me forever to set this up but this helped https://docs.ray.io/en/master/ray-overview/installation.html#m1-mac-apple-silicon-support looks like you need to be super careful about your environment with Ray and grpcio package - do you use Conda? on which machine do you run it?
c
Tried specifying the SubprocessFlowRunner with the virtual env path in my case. but did not help
I am running a venv on a linux ubuntu machine
a
yup you're right, I was able to reproduce your error, thanks for reporting that @Marvin open "Flow run with Ray and agent throws: `ImportError: cannot import name 'SubprocessFlowRunner' from partially initialized module 'prefect.flow_runners'`"
m
c
thanks for the quick support as always
🙌 1
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by