Hey everyone, on my local agent everything works fine. But if i want to use 

DockerRun(image="prefecthq/prefect")

  with

prefect agent docker start

 i´m getting this Error:

Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")

I know that i can´t import other dependencies but i only import following modules from prefect in the script:

from prefect import Flow, task

from prefect.run_configs import LocalRun,DockerRun

Or is it possible to run 

DockerRun()

 in an existing Container?

Hi @Tom Tom - any modules/packages needed to run your flows in a container will either need to be pip-installable by the container or manually placed there during the build process. You can accomplish this by either publishing modules or creating a custom image to use for your flows

Ah ok, i thought prefect is already installed in image

"prefecthq/prefect"

. I´ll build a custom image on base on

"prefecthq/prefect"

with the prefect dependencies, hope it works 🙂

Oh I'm sorry I may have misunderstood - the prefect image should have prefect dependencies installed

Oh, ok, well that´s my python script for testing:

from prefect import Flow, task
from prefect.run_configs import LocalRun,DockerRun

#classes
class eins:
    @task
    def function():
        return 1

Test=eins()

#Flow
with Flow("Prefect Test Flow") as flow:
    a = Test.function()

#flow.run_config = LocalRun()
flow.run_config = DockerRun(image="new")
flow.register(project_name="test")

I start my server on CMD with

prefect server start

and my docker agent also on CMD with

prefect agent docker start --label Andromeda

. Andromeda is my PC-Name, if I don´t define it, the agent will not run anything and the tasks will be "late" but not executing.

Hey, I think what is happening is that your flow is using LocalStorage by default, which writes the Flow in

~/.prefect

by default when you register. I think you are on a Windows machine so your home evaluates to

C:/users/something

. And then when you use

DockerRun

it imports your flow from

LocalStorage

and looks for it under the

~.prefect

folder. But you don’t have

C:/users/something

.

Hey @Kevin Kho , thank you very much for your help. I´ll tried several path combinations but no success, i´ll try it tomorrow again and will reply, maybe i´m just to tired at 3:40 AM 😅

Can I see your Dockerfile and your flow?

I just use

run_config=DockerRun(image="prefecthq/prefect")

, so i use the prefect standard docker image and i dont have any Dockerfile(i think?).

with Flow("Prefect_Test_Flow",storage=Local(path="/Desktop/test.py",stored_as_script=True), run_config=DockerRun(image="prefecthq/prefect")) as flow:
    a = Test.function()

The

Local(path="Desktop/test.py")

will look for that path inside the container so you need to get the file inside the container somehow.

creates

run_config=DockerRun(image="prefecthq/prefect")

a container? because at the moment no container is running.

The docker agent will pull

prefecthq/prefect

and then look for the local path (relative to the container), and then load the flow and run it on top. This path thing is trickier with

Local

but with something like

GitHub

or

S3

you don’t need to worry about that script being stored in the image since it will be downloaded during runtime.

Local

just needs it to be in the local directory. If you are using

DockerRun

, it’s a path in the container. If you are using

LocalRun

, it’s a path on the host machine.

Ah ok i can follow you, e.g. with

docker cp <src-path> <container>:<dest-path>

  i need to copy the file into the container, so the agent find/load and can run the flows. But this is tricky cause during runtime ofc i cant use comands, maybe i can start a container manually, copy file, and then run the flow with

DockerRun()

and pass them the container ID?

I think you would need to make a new image because I think Prefect starts a container as it calls

dockerpy's

create_container

so I don’t believe you can just pass a container to DockerRun. DockerRun also does not take a container id as an argument.

ok, great that was my second idea to create an own image based on prefecthq/prefect. I´ll try that out 😃

Nice work!

ok, one additional question. If i add 2 or more .py files in the dockerfile, can i use in

DockerRun()

more than one path/file? Or 1 flow 1 file?

So 1 flow is limited to one file. That place the flow is stored is a

Storage

concept rather than the

RunConfiguration

, but yes you can include multiple files in the container. Check this for more information on Docker storage. If those are files you need to import, it’s best to install as a Python module. If those are files you read in like

.sql

or

.yaml

then just including them in the container for your Flow to use will work.

U´re the best, thank you very much and have a good day 👏