Hey there. I’ve been playing around with Docker storage today, trying to get all source code packaged together with the flows each time they are registered, and am using the

files

and

env_vars

attributes as outlined in the Docs. But it seems that my

.dockerignore

file (in the directory from which I am registering the flow) is ignored by this build process. I have massive data directories nested at different levels inside the codebase so it’s essential there be a way to include this somehow. Anyone come across this / have any ideas? I’m curious about playing around with a custom Dockerfile, but the code that copies flows to a tmp Dockerfile in

create_docker_file

is dynamic so it seems tough that I could write a standalone Dockerfile to handle this case.

Hey @Michael, this is the first I’ve seen this. I’ll look into it.

Hey @Michael, can you try including

.dockerignore

in the

files

arg for the storage?

Interesting… I’ll check it out 🙂

Perhaps the build context is not where I think it is

Yes! (unless you've looked through the internals of Docker storage) When building Docker storage, Prefect creates a temporary directory that contains a new Dockerfile (created by Prefect) and also copies over all your files. For the

.dockerignore

to be respected, it will need to be copied into the root of this temporary directory https://github.com/PrefectHQ/prefect/blob/master/src/prefect/storage/docker.py#L435

Ah well, there’s the answer! But yes I’m also not immediately sure how to handle that nicely. I think that a .dockerignore path to go with the dockerfile one would be cool 🤷 but it looks unfixable without changing the Prefect source

Yeah I can open an issue for this regardless of workarounds will it work if you specify

files

like so?

files = {
    ...
    "/path/to/.dockerignore": ".dockerignore"
}

Unfortunately not, because that just creates a COPY statement inside the generated

Dockerfile

that copies the

.dockerignore

inside the image, whereas the docker build process needs to read the

.dockerignore

file before starting to build (at least, this is how I understand why it doesn’t work)

@Marvin open "Add option to

Docker

storage to respect

.dockerignore

file if using

files

argument"

@Zach Angell just wanted to confirm that the workaround I was attempting also doesn’t work; the

create_dockerfile_object

copies the specified files into a tmp dir as well as generating the

Dockerfile

text. So the docker build isn’t the long part, it’s just a massive

shutil.copytree

call that copies my files (the entire current directory) into the tmp directory inside the current directory. It’s frustrating because in my case I shouldn’t have to perform the copy since the files are already within the build context, but I get that it’s designed to allow a user to specify files anywhere, hence the copy to make sure.

Shoot okay, sorry about that. If you're willing to take a shot at a PR, I'd encourage it. Otherwise, I'd keep an eye on the issue as members of our team or the community pick it up. Thanks for all your help in debugging here! I'm happy to help out with any workarounds in the meantime, but nothing jumps out to me