I have a current organization of Dockerfile package init py Prefect Community #ask-community

I have a current organization of: - Dockerfile - p...

Jason

04/20/2022, 8:30 PM

I have a current organization of: • Dockerfile • package ◦ init.py with DockerStorage tied to ECR • projects ◦ example ▪︎ flow.py Given this, I keep running into the fact that package can't find the Dockerfile in the root directory. What's the context for the flows that are registered? I tried using relative pathing, (i..e, "../Dockerfile") but I'm not sure what the base directory is. I could try just moving the Dockerfile into package in the same directory as init.py, but I'm trying to understand what I'm doing wrong here.

Kevin Kho

04/20/2022, 8:35 PM

Are you using Docker storage like this? I think in general though you can’t build from a folder above? I might be wrong though. Very unconfident on that.

Jason

04/20/2022, 8:43 PM

I also tried moving the Dockerfile to package so that it's in the same as

__init__.py

and the same issue happens. I'd like to avoid having to have a Dockerfile per project or something.

Jason

04/20/2022, 8:47 PM

The only way I could import the Config class across flows was to build and install the package with setup.py:

Copy code

def storage(cls):
        if getenv("ENV") == "dev":
            # from prefect.storage import Local

            # return Local()
            from prefect.storage import Docker

            return Docker()
        else:
            from prefect.storage import Docker

            return Docker(
                registry_url=f"{Secret('AWS_ACCOUNT_ID').get()}.<http://DKR.ECR.US-EAST-1.AMAZONAWS.COM|DKR.ECR.US-EAST-1.AMAZONAWS.COM>",
                dockerfile="Dockerfile",
                image_name="platform-prefect",
                image_tag="latest",
                path="../",
            )

I wonder if the path since it's imported to the flow is relative to the flow itself?

Kevin Kho

04/20/2022, 9:27 PM

Can you try using ../Dockerfile instead of using the path there?

Kevin Kho

04/20/2022, 9:28 PM

path is a path to the flow file in the image if you use

stored_as_script=True

Jason

04/21/2022, 2:39 PM

Thanks - So the solution (I think) was to actually delete the path and dockerfile args and let it inherit the default. Same filename - same location - but now works. 😕

Kevin Kho

04/21/2022, 2:42 PM

path is the path for the Flow file though inside the container. I think you were misusing the path keyword there

Kevin Kho

04/21/2022, 2:42 PM

Dockerfile args might work

4 Views

Open in Slack

Previous Next