Hi folks, Have an issue with running a flow using ...
# prefect-server
Hi folks, Have an issue with running a flow using a Kubernetes Agent, and a Module storage. So here is what I do. The agent docker image is based on prefecthq/prefect:0.14.19, and in my dockerfile, I have the following folder hierarchy:
Copy code
in my_flow.py, imagine something like that:
Copy code
def get_flow():
    with Flow('my_flow', storage=Module("B.flows.my_flow:get_flow")) as flow:
    return flow
In the dockerfile, I ensure that the
env var holds the folder
so that I can import
setting this at the end of the DockerFile:
Copy code
After that I register this flow with a
and a `DaskExecutor`:
Copy code
flow.run_config = KubernetesRun()
flow.executor = DaskExecutor()
Then from a prefect server, I trigger the flow run, but I get the following error:
Copy code
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'A'")
I don't really get why my module is not found. I have tested a local deployment using docker-compose with a local agent (running in the compose stack), and it was working fine. So I am missing something here. Any ideas?
Shall the module be also available on the prefect server apollo pod?
Hi @Romain! Yes you need it installed and available in the execution location.
But I thought that in prefect , the server was somewhat agnostic of the execution code. And only the agent was required to know the code. How would it work if I would be using prefect cloud and a Module storage then?
With Docker storage, the container is downloaded and executed on the agent. For Module storage, the module would need to exist on the execution side also because it doesn’t download it from a location. This is true for Cloud as well.