hey guys, i'm trying to get prefect running on kubernetes. i've packaged my entire python package into docker container, and i'm able to run the flows in the container just fine locally using

docker run

. however, when i deploy the flows to kube and use prefect cloud to trigger a run, i get the following error:

Failed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n  ModuleNotFoundError("No module named \'my_package\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')

i'm using GCS as the storage. i'm also wondering if/why i need to use GCS as a storage -- doesn't the execution environment have access to the flows? why do they even need to be pickled? thanks in advance! 🙂

This page provides more details: https://discourse.prefect.io/t/when-i-run-my-flow-i-see-an-error-failed-to-load-and-exe[…]derror-no-module-named-users-username-what-is-happening/33 In short: your Docker container usually has code dependencies (e.g. your custom modules or things like requests or pandas), while storage references your flow file

thanks anna! are tasks serialized too or are they considered dependencies that should be deployed into the execution environment?

Tasks are not serialized, just the metadata around them, but the code is loaded in from the Storage. What RunConfig are you using? DockerRun?

oh, i didn't specify any RunConfig

So what is your execution environment? Local agent?

kubernetes

So it’s using the default Prefect image to run the Flow?

i have a Dockerfile that uses prefecthq/prefect:latest as the base image

Where did you pass the Dockerfile? I thought you where on GCS Storage?

the dockerfile builds all my code into a python package. that is deployed to kubernetes

You need to supply that image to

KubernetesRun

otherwise it won’t know what image to use and will pull the default Prefect image and run your Flow on top of that. The Flow does not run in the same pod as the agent. The agent spins up a new Kubernetes job so the execution environment is different

Ah i see, thanks for the clarification. I will make that change and let you know the outcome. Thanks!