Hey guys, normally I register flows on the VM runn...
# ask-community
j
Hey guys, normally I register flows on the VM running the local agent. If I try to register a flow to the Prefect cloud from another machine, I get a connection error (see screenshot). Any ideas?
a
It seems that you try to register your flow to Prefect Cloud, but the error message looks like you would use Server. Could you run
prefect diagnostics
and check "prefect_backend"? If it turns out you use Server, you can then switch the backend using:
prefect backend cloud
j
Awesome, thanks!
The flows registered from the non-agent system are not running properly. I'm getting a "Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")". Do you happen to know if it's because I still missed something with the registering?
a
No this is not related to registration. You seem to be using some Python package in your flow, which is not part of Prefect, and you need to install it in your Flow's environment. What kind of Storage do you use?
j
I didn't change storage type so I assume the standard. If I manually execute the flow from the VM where the local agent runs it works (no packages missing).
Hmmm, it seems to work now. Although I have no clue what I did different 🙂 ...
👍 1
a
Probably what you did differently is running the flow in the environment, where the missing package is no longer missing 🙂
r
Hi, I'm facing the same import error issue. ("Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")"). When I .run() the python file on my personal machine and the python file on my virtual machine, the code works both ways. When I register to prefect cloud from my virtual machine, running the flow from the cloud platform works. However when I register from my personal machine and try to run the flow from the cloud, I receive this importerror. Any ideas ?
a
Hi @Ruben Sik. Did you install the package within the environment you run the flow in? Can you share your run configuration?
r
Hi Anna, the run config is quite default:
and a testscript @Anna Geller without any fancy imports. This with the same error.
a
Could you try moving the flow object outside of the build_flow() function, as follows:
Copy code
from prefect import task, Flow


@task
def hello_world():
    return "hello world"


with Flow("test-flow") as flow:
    hello_world()

if __name__ == "__main__":
    flow.register()
Otherwise, since
prefect
is the only module you import, can you check if prefect is installed in the environment where you run this?
r
Hi @Anna Geller, with that code adjustment the error still persists. On both my local machine and virtual machine prefect is installed and running the flow works (via. .run()). Are you referring to a different environment that I should check?
a
@Ruben Sik I see you added the label "data-gateway" on your UniversalRun run configuration. Did you also set the same label on your agent? Which command did you use to start the agent?
r
@Anna Geller I've started the agent with the default command. 'python ...../local_agent.py' after which it starts with:
a
To replicate this, I started the agent using
prefect agent local start --label "data-gateway"
. Then I registered this flow by explicitly specifying LocalRun with your label, and it seems to work:
Copy code
from prefect import task, Flow
from prefect.run_configs import LocalRun


@task
def hello_world():
    return "hello from local agent"


with Flow("test-flow", run_config=LocalRun(labels=["data-gateway"])) as flow:
    hello_world()

if __name__ == "__main__":
    flow.register("DataScience")
so I believe there is some mismatch in the run configuration or Agent environment. Could you check that and try again?
k
Hey, the
ModuleNotFoundError("No module named 'C'")".
happens when you use the default Local storage when you register, and then try to run the Flow in a different machine. When you register, Prefect saves the Flow in storage. When you run the flow, Prefect loads it from storage and executes it. For the default Local storage, this is saved in the
.prefect
folder when you register. So when Prefect executes the content of the Flow, it imports it from Storage. The
No module named 'C'
is because you’re using Windows, and that is the default drive and the import on the Prefect end will look something like “from C..prefect.flows import myflow” (this is not super accurate). This specific error is just saying it can’t find the file to load in. This happens when you register with local on one machine then run it in another because the file is not there.
upvote 1
🙌 1
r
Hi @Anna Geller, I've tried this but unfortunately the error is still there. @Kevin Kho Would this mean that this error could be solved by setting storage as blob storage ? And accessing this storage from both machines? https://docs.prefect.io/orchestration/execution/storage_options.html#azure-blob-storage
k
Yes exactly. You need a storage that both machines can pull from
r
Allright. Thanks for the explanation. Makes sense. I think i'm almost there now 🙂One last question, I've connected to blob storage and my flow seems to store on Azure blob storage. Calling the agent from the cloud now results in the error 'attributeError("'NoneType' object has no attribute rstrip'. I've set the AZURE_STORAGE_CONNECTION_STRING as environment variable on both machines. Any ideas why the agent on the other machine would be unable to read the connection string?
k
What Prefect version are you on?
r
Hi @Kevin Kho, my personal machine was running on prefect 0.15.3. The virtual machine on 0.15.4. I've updated both to 0.15.6. I've used the secret stored for the connection and it works! For future reference my working code is added in the image below. @Anna Geller & @Kevin Kho 🙌
👍 1
k
Glad you got that working!