https://prefect.io logo
c

Carlos Paiva

02/03/2022, 6:04 PM
Hi guys, How can I get the working directory or file path for Docker Run with GitHub Storage? I need to import some utilities from my repo, but I keep getting import errors.
k

Kevin Kho

02/03/2022, 6:35 PM
Github storage is not meant for that because even if the repo is cloned, the path where the flow is running is different. We have had people try to add to the path but it hasn’t worked. I think there some options: 1. Make your Github library pip installable, then clone and pip install that yourself in the execution environment 2. DockerRun can take
EXTRA_PIP_PACKAGES
in the
env
and the entrypoint will install it. I think you can pass a git repo link if pip installable.
c

Carlos Paiva

02/03/2022, 6:55 PM
@Kevin Kho Unfortunately that will not work as the repository has sensitive data and I can’t publish it on PyPI.
k

Kevin Kho

02/03/2022, 6:57 PM
Does your org have a private repository like Artifact? Or you can do something like this
I don’t mean upload to PyPI by pip installable. I just mean having the
setup.py
so you can install after cloning like
pip install -e .
. You know what I mean?
c

Carlos Paiva

02/03/2022, 7:29 PM
We don’t have Artifact. I have tried to add
<git+ssh://git@github.com/myuser/repo.git>
to
EXTRA_PIP_PACKAGES
but the flow runs keep stuck on submitted.
k

Kevin Kho

02/03/2022, 7:31 PM
How did you authenticate your agent? and you added the
setup.py
?
c

Carlos Paiva

02/03/2022, 7:38 PM
The agent is running with the `prefect agent docker start`and the setup.py is in the project/repo.
k

Kevin Kho

02/03/2022, 7:47 PM
Ah ok sorry if that doesn’t work smoothly, we likely really need those dependencies inside the container as well. Some people use CI/CD to push their container when the repo changes and the the Flow will be responsible for grabbing it
c

Carlos Paiva

02/03/2022, 7:49 PM
How can I do that?
k

Kevin Kho

02/03/2022, 7:50 PM
Which part? The CI/CD or putting the stuff in the container?
c

Carlos Paiva

02/03/2022, 7:52 PM
Adding the stuff in the container
k

Kevin Kho

02/03/2022, 7:52 PM
The Docker here. For CI/CD, that is too involved for me to advice because it’s specific to your repo and set-up
Also though, this discussion might help you
c

Carlos Paiva

02/04/2022, 10:17 AM
Does Local Storage solve the problem?
k

Kevin Kho

02/04/2022, 1:16 PM
Not Local Storage but LocalRun can solve it if all the dependencies are available on the execution side