Selim Raboudi
09/11/2025, 4:53 PM# Prefect deployment configuration for ingest-mongotoy workflow
# Generated using Python
build: null
push: null
pull: null
deployments:
- name: deployment-ingest-mongotoy-prod-daily
flow_name: ingest-mongotoy
entrypoint: workflows/ingestion/ingest-mongotoy/src/ingest_mongotoy/_flows.py:flow_main
pull:
- prefect.deployments.steps.git_clone:
id: clone-step
repository: <https://github.com/myorg/my-repo.git>
branch: main
credentials: '{{ prefect.blocks.github-credentials.my-repo }}'
- prefect.deployments.steps.run_shell_script:
id: install-uv
script: pip install uv
- prefect.deployments.steps.run_shell_script:
id: seed-git-creds
# This avoids prompting and makes any git+https fetch use the token
script: |
git config --global url."<https://x-access-token>:${GITHUB_TOKEN}@github.com/".insteadOf "<https://github.com/>"
git config --global url."<https://x-access-token>:${GITHUB_TOKEN}@github.com/".insteadOf "<ssh://git@github.com/>"
git config --global url."<https://x-access-token>:${GITHUB_TOKEN}@github.com/".insteadOf "git@github.com:"
env:
GITHUB_TOKEN: "{{ prefect.blocks.github-credentials.my-repo.token }}"
expand_env_vars: true
- prefect.deployments.steps.run_shell_script:
id: uv-sync
script: 'uv export --format requirements.txt --locked --no-dev --output-file
requirements.txt
uv pip install --system -r requirements.txt
uv pip install --system .'
directory: '{{ clone-step.directory }}/workflows/ingestion/ingest-mongotoy'
work_pool:
name: managed-workpool
work_queue_name: null
job_variables:
image: prefecthq/prefect:3-python3.11
It seems the seed-git-creds is not working.
I get the following error:
fatal: could not read Username for '<https://github.com>': terminalMarvin
09/11/2025, 4:54 PMMarvin
09/11/2025, 4:57 PMuv pip install of your private dependency, so the URL rewrite isn’t being applied.
A couple of likely issues and fixes:
1) Make sure the seeding script uses the correct format (no angle brackets, correct username:password placement)
Slack auto-adds <…> around URLs, but in your YAML those must not be present. Also, the username/password separator needs to be a colon after the username. This is the most robust form:
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "<https://github.com/>"
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "<ssh://git@github.com/>"
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "git@github.com:"
Notes:
- No angle brackets anywhere
- The base ends with a trailing slash
- Username is x-access-token, followed by a colon, then the token
2) Consider using a .netrc file instead (very reliable)
This avoids the URL rewrite entirely and is often simpler for VCS installs:
printf "machine <http://github.com|github.com>\n login x-access-token\n password %s\n" "$GITHUB_TOKEN" > ~/.netrc
chmod 600 ~/.netrc
3) Prevent interactive prompts during install
Add this to your uv step environment to fail fast if credentials aren’t picked up:
GIT_TERMINAL_PROMPT: "0"
Putting it together, here’s a cleaned-up version of your steps:
- prefect.deployments.steps.git_clone:
id: clone-step
repository: <https://github.com/myorg/my-repo.git>
branch: main
credentials: '{{ prefect.blocks.github-credentials.my-repo }}'
- prefect.deployments.steps.run_shell_script:
id: install-uv
script: pip install uv
# Option A: URL rewrite
- prefect.deployments.steps.run_shell_script:
id: seed-git-creds
script: |
set -eu
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "<https://github.com/>"
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "<ssh://git@github.com/>"
git config --global url."<https://x-access-token:${GITHUB_TOKEN}@github.com/>".insteadOf "git@github.com:"
env:
GITHUB_TOKEN: "{{ prefect.blocks.github-credentials.my-repo.token }}"
expand_env_vars: true
# Option B: .netrc (use this instead of the URL rewrite block above)
# - prefect.deployments.steps.run_shell_script:
# id: seed-netrc
# script: |
# set -eu
# printf "machine <http://github.com|github.com>\n login x-access-token\n password %s\n" "$GITHUB_TOKEN" > ~/.netrc
# chmod 600 ~/.netrc
# env:
# GITHUB_TOKEN: "{{ prefect.blocks.github-credentials.my-repo.token }}"
# expand_env_vars: true
- prefect.deployments.steps.run_shell_script:
id: uv-sync
script: |
uv export --format requirements.txt --locked --no-dev --output-file requirements.txt
uv pip install --system -r requirements.txt
uv pip install --system .
directory: '{{ clone-step.directory }}/workflows/ingestion/ingest-mongotoy'
env:
GIT_TERMINAL_PROMPT: "0"
A couple of additional checks:
- What is the exact VCS URL in your pyproject/requirements (e.g., git+<https://github.com/org/private.git@tag> vs `git+ssh://git@github.com/org/private.git@tag`)? If it’s using SSH, the rewrite above should still cover it, but using git+https is simpler with tokens.
- Avoid echoing git config -l in logs — it will leak the token.
If you apply the rewrite exactly as shown (or the .netrc method) and set GIT_TERMINAL_PROMPT=0, uv pip install should stop prompting and pull your private dependency non-interactively.
References:
- Git url.insteadOf docs: https://git-scm.com/docs/git-config#Documentation/git-config.txt-urlltbasegtinsteadofMarvin
09/11/2025, 4:57 PMBring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by