I'm having issues getting a flow to run locally us...
# ask-community
r
I'm having issues getting a flow to run locally using Prefect Server, Docker Agent, DockerRun, and Script Storage (via gitlab) (I think that is an ok combo...). When the flow is triggered, the container is built, but the agent is unable to pull the flow from gitlab
gitlab.exceptions.GitlabGetError: 404: 404 Not Found.
This is a self-hosted gitlab, but a public repo. Does this error suggest authentication issues or am I missing something else? Thanks! agent is launched with:
prefect agent docker start --api <http://localhost:4200> --show-flow-logs --label docker_local
storage configs:
GitLab(host = selfhost_gitlab_url, repo='rgaffney/simple_orchestration', path='HelloWorld_flow.py',ref='prefect_v1')
run_configs:
DockerRun(image="prefecthq/prefect:core-0.15.5", env={"EXTRA_PIP_PACKAGES": 'asyncio aiohttp aiofiles prefect[gitlab]'}, labels=['docker_local'])
a
Hi @Rowan Gaffney, here are some ideas to try: 1. Are all those components running on a local machine or on a remote instance such as AWS EC2? This could be relevant to check whether networking might be the issue (e.g. your agent not being able to communicate with Gitlab because you can’t reach the Gitlab Server from that compute instance). 2. Otherwise, this can indeed have something to do with authentication. Does your Docker Agent have access to pull this Gitlab repository? To check that, you could try cloning the repo from the same instance where your agent is running. You might need a personal access token for this. Then you can test it this way:
Copy code
curl <<https://gitlab-ci-token>:zzzzzzzzzzzzzzzzzz@gitlab.example.com/api/v4/projects/215/repository/tags>
or
Copy code
curl --header "PRIVATE-TOKEN: zzzzzzzzzzzz" https://...
3. Another thing worth checking: how do you start your Server? From 0.15.5 on (the version you are using!) it requires the
--expose
flag to expose it to all interfaces. Docs here.
k
I think this might be a relevant issue.
python-gitlab 2.7
and above gives these errors
upvote 1
r
Thanks for the suggestions @Kevin Kho, and @Anna Geller! I tried with the older versions of python-gitlab, but no luck. Everything is local, so no EC2 or networking issues. However, I was able to get it to work using the Project ID (as an int, not a string), rather than the repo name. Really like the idea of "script storage" for flows. Perhaps the docs could use some more clarifications, but also could simply be "user error" on my part. Thanks again for the prompt responses!
👍 1