j

    Jelle Vegter

    11 months ago
    Hey guys, normally I register flows on the VM running the local agent. If I try to register a flow to the Prefect cloud from another machine, I get a connection error (see screenshot). Any ideas?
    Anna Geller

    Anna Geller

    11 months ago
    It seems that you try to register your flow to Prefect Cloud, but the error message looks like you would use Server. Could you run
    prefect diagnostics
    and check "prefect_backend"? If it turns out you use Server, you can then switch the backend using:
    prefect backend cloud
    j

    Jelle Vegter

    11 months ago
    Awesome, thanks!
    The flows registered from the non-agent system are not running properly. I'm getting a "Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")". Do you happen to know if it's because I still missed something with the registering?
    Anna Geller

    Anna Geller

    11 months ago
    No this is not related to registration. You seem to be using some Python package in your flow, which is not part of Prefect, and you need to install it in your Flow's environment. What kind of Storage do you use?
    j

    Jelle Vegter

    11 months ago
    I didn't change storage type so I assume the standard. If I manually execute the flow from the VM where the local agent runs it works (no packages missing).
    Hmmm, it seems to work now. Although I have no clue what I did different 🙂 ...
    Anna Geller

    Anna Geller

    11 months ago
    Probably what you did differently is running the flow in the environment, where the missing package is no longer missing 🙂
    r

    Ruben Sik

    11 months ago
    Hi, I'm facing the same import error issue. ("Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")"). When I .run() the python file on my personal machine and the python file on my virtual machine, the code works both ways. When I register to prefect cloud from my virtual machine, running the flow from the cloud platform works. However when I register from my personal machine and try to run the flow from the cloud, I receive this importerror. Any ideas ?
    Anna Geller

    Anna Geller

    11 months ago
    Hi @Ruben Sik. Did you install the package within the environment you run the flow in? Can you share your run configuration?
    r

    Ruben Sik

    11 months ago
    Hi Anna, the run config is quite default:
    and a testscript @Anna Geller without any fancy imports. This with the same error.
    Anna Geller

    Anna Geller

    11 months ago
    Could you try moving the flow object outside of the build_flow() function, as follows:
    from prefect import task, Flow
    
    
    @task
    def hello_world():
        return "hello world"
    
    
    with Flow("test-flow") as flow:
        hello_world()
    
    if __name__ == "__main__":
        flow.register()
    Otherwise, since
    prefect
    is the only module you import, can you check if prefect is installed in the environment where you run this?
    r

    Ruben Sik

    11 months ago
    Hi @Anna Geller, with that code adjustment the error still persists. On both my local machine and virtual machine prefect is installed and running the flow works (via. .run()). Are you referring to a different environment that I should check?
    Anna Geller

    Anna Geller

    11 months ago
    @Ruben Sik I see you added the label "data-gateway" on your UniversalRun run configuration. Did you also set the same label on your agent? Which command did you use to start the agent?
    r

    Ruben Sik

    11 months ago
    @Anna Geller I've started the agent with the default command. 'python ...../local_agent.py' after which it starts with:
    Anna Geller

    Anna Geller

    11 months ago
    To replicate this, I started the agent using
    prefect agent local start --label "data-gateway"
    . Then I registered this flow by explicitly specifying LocalRun with your label, and it seems to work:
    from prefect import task, Flow
    from prefect.run_configs import LocalRun
    
    
    @task
    def hello_world():
        return "hello from local agent"
    
    
    with Flow("test-flow", run_config=LocalRun(labels=["data-gateway"])) as flow:
        hello_world()
    
    if __name__ == "__main__":
        flow.register("DataScience")
    so I believe there is some mismatch in the run configuration or Agent environment. Could you check that and try again?
    Kevin Kho

    Kevin Kho

    11 months ago
    Hey, the
    ModuleNotFoundError("No module named 'C'")".
    happens when you use the default Local storage when you register, and then try to run the Flow in a different machine. When you register, Prefect saves the Flow in storage. When you run the flow, Prefect loads it from storage and executes it. For the default Local storage, this is saved in the
    .prefect
    folder when you register. So when Prefect executes the content of the Flow, it imports it from Storage. The
    No module named 'C'
    is because you’re using Windows, and that is the default drive and the import on the Prefect end will look something like “from C..prefect.flows import myflow” (this is not super accurate). This specific error is just saying it can’t find the file to load in. This happens when you register with local on one machine then run it in another because the file is not there.
    r

    Ruben Sik

    11 months ago
    Hi @Anna Geller, I've tried this but unfortunately the error is still there. @Kevin Kho Would this mean that this error could be solved by setting storage as blob storage ? And accessing this storage from both machines? https://docs.prefect.io/orchestration/execution/storage_options.html#azure-blob-storage
    Kevin Kho

    Kevin Kho

    11 months ago
    Yes exactly. You need a storage that both machines can pull from
    r

    Ruben Sik

    11 months ago
    Allright. Thanks for the explanation. Makes sense. I think i'm almost there now 🙂One last question, I've connected to blob storage and my flow seems to store on Azure blob storage. Calling the agent from the cloud now results in the error 'attributeError("'NoneType' object has no attribute rstrip'. I've set the AZURE_STORAGE_CONNECTION_STRING as environment variable on both machines. Any ideas why the agent on the other machine would be unable to read the connection string?
    Kevin Kho

    Kevin Kho

    11 months ago
    What Prefect version are you on?
    r

    Ruben Sik

    11 months ago
    Hi @Kevin Kho, my personal machine was running on prefect 0.15.3. The virtual machine on 0.15.4. I've updated both to 0.15.6. I've used the secret stored for the connection and it works! For future reference my working code is added in the image below. @Anna Geller & @Kevin Kho 🙌
    Kevin Kho

    Kevin Kho

    11 months ago
    Glad you got that working!