When using gitlab storage am I right in thinking o...
# ask-community
y
When using gitlab storage am I right in thinking only the flow is cloned down to a remote agent running the flow? Or does the whole repo get cloned... I can't see it in the docs
Specifically GitLab, I can see the base git storage does load the whole repo
k
The whole repo is cloned, but this is meant to pull SQL or YAML files. It’s not meant to execute other Python modules because it doesn’t pip install the repo. If you have other dependencies, using Docker is the recommendation
1
upvote 1
a
Correct, you shouldn’t rely on the presence of other modules being cloned from the repository because Prefect executes only the flow and even if other files were cloned as well, Prefect wouldn’t know how to install those dependencies. That’s why it’s best to package your dependencies either into: 1. An installable python package that gets installed in your execution environment 2. Docker image
1
y
Thanks for the advice, but for other files that are part of the project for example a main.py and a config.py containing just env vars, then cloning should be fine right? Main.py can import all python files in the same directory as itself
k
The working dir path is different when you the Flow is running and you are cloning the repo. Some users have tried to hack it with a
sys.path.append
but I haven’t seen it work. It’s pretty tricky
y
Hmm I see, i thought by default cwd is added to the python path, so no matter where it's cloned main.py would be able to import other .py files that share the same directory
I think I might be misinformed about the python path thing. Might be able to use relative imports but yeah you're right it's tricky and potentially messy