:wave: I don't understand how to create a deploym...
# prefect-community
m
👋 I don't understand how to create a deployment where the code is in a subdirectory in GitLab/Github
Copy code
my_project.git
/src/
/src/dbt/
/src/dbt/Earthfile # Equivalent to Dockerfile
/src/dbt/my_prefect_job.py
/src/dbt/...
And my Earthfile is like
Copy code
FROM prefecthq/prefect:2.7-python3.10
WORKDIR /code
...
COPY . .
And so
my_prefect_job.py
is under
/code/my_prefect_job.py
in my docker image and under
/src/dbt/
in my gitlab repository But once deployed, the run output the following
Copy code
Downloading flow code from storage at 'src/dbt'

Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1016, in get_code
  File "<frozen importlib._bootstrap_external>", line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/code/prefect_job.py'
And so I don't understand why it cannot find my flow in gitlab under the right subdirectory nor the flow in the docker contain under the right path 🤷 Did not find any good example about github/gitlab subdir storage strategy. Thanks in advance 🙏
a
I'm curious what would happen if you
COPY . /opt/prefect/flows/
instead?
m
I can try but I don't understand why it should work as I specified the WORKDIR in my Dockerfile
c
my guess would be because that’s not the entrypoint for the flow, and isn’t part of the pythonpath
Not sure if this is relevant, or oversight, but also:
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '/code/prefect_job.py'
has a different file name than what you reference:
/src/dbt/my_prefect_job.py
plus one 1
m
What is weird is that
/code/prefect_job.py
exists in my docker image while
/src/dbt/my_prefect_job.py
exists in my repository And I don't know for which source (docker image or repository) this FileNotFoundError refers to
I removed the WORKDIR in my docker image and I still get the same error
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '/code/prefect_job.py'
I cannot find any mention of
/code/prefect_job.py
in my repo. This is really weird
I got it working by not using any subfolder ... But it's not great, this is like forcing me to have one repo per project and not the flexibility to have sub projects within a repo 😢
c
You should have
path
and
entrypoint
- path is the relative path of your remote storage, and entrypoint is the file/flow defined. https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments.py#L240 So i’d expect something like
path=/src/dbt/
and
entrypoint=my_prefect_job.py:<flowname>
To be fair though, you are right, there isn’t good documentation on this at the moment. It’s on my plate to add
b
Not sure if this helps but I had that same issue here and solved it by making sure the github relative path matches the entrypoint path.
Copy code
prefect deployment build -a data-products/src/orchestration/main_orchestration_flow.py:main_orchestration_flow -sb github/data-flows-dev/data-products/src/orchestration --name base
m
Yes I made it work with something similar 🙌