Hi. I’m new to Prefect and try to deploy a GreatExpectations environment into my docker/Kubernetes deployment. Unfortunately the only way I could get files into the container until now was by using the “files” parameter in the “Docker” storage class. To fill my container this way seems a bit odd to me. Is there another way I can just COPY my files into my container environment?

Hi Rolf, Ignoring GreatExpectations, if you want to get extra files into a Prefect-built docker image you have a few methods: • Pass in the

files

parameter to the

Docker

class • Pass a custom

dockerfile

that manually adds your files to the

Docker

class • Pre-build a new base image that already has your files added, then pass in that image as

base_image

to

Docker

.

Thanks for the answer @Jim Crist-Harif. The “manual passing of a docker file” was the feature I was looking towards. I tried now relativ and absolute paths with the docker “COPY” command, but unfortunately Prefect still adds the prefix “/var/lib/docker/tmp/docker-builder994084773” before the path, resulting in an Error.

I tried now relativ and absolute paths with the docker “COPY” command, but unfortunately Prefect still adds the prefix “/var/lib/docker/tmp/docker-builder994084773” before the path, resulting in an Error.

Sorry, are you saying this happened with a custom dockerfile? Or using the

files

arg to

Docker

?

until now I’ve done both. using the “files” arg and the “Docker” one. • “Docker” the prerequisites of “pyodbc” • “files” for the manual copying.

Sorry, I'm unable to replicate, can you provide an example (something simple) that doesn't work the way you expect?

I was looking to just copy a folder with all its content with the dockerfile. unfortunately that is not working due to a prefix which is added.

Hmmm, I'm unable to replicate.

$ tree
.
├── Dockerfile
├── flow.py
└── samplefolder
    └── testfile

1 directory, 3 files

$ cat Dockerfile
FROM prefecthq/prefect:latest-python3.8

COPY samplefolder/. /opt/app/great_expectations/

$ cat flow.py
from prefect import Flow, task
from prefect.environments.storage import Docker

@task
def hello():
    print("hello")

with Flow("example") as flow:
    hello()

flow.storage = Docker(dockerfile="Dockerfile")

flow.register("testing")

$ python flow.py
...
Successfully built fcc4916c9281
Successfully tagged example:2020-12-15t16-23-44-416437-00-00
Flow URL: <https://cloud.prefect.io/jim-prefectio/flow/1debc0b1-aa7b-42b1-b6df-97051786279c>
 └── ID: 38ad0ce0-f222-432e-860e-c2fec8ac21eb
 └── Project: testing
 └── Labels: []

Thanks @Jim Crist-Harif I’ll debug my way from here.