Fugo Takefusa
09/22/2025, 2:03 AMMarvin
09/22/2025, 2:03 AMMarvin
09/22/2025, 2:08 AMgit@github.com:org/repo.git
- Ensure the worker’s environment has SSH set up (private key available, optional ssh-agent, and known_hosts
for github.com)
- In prefect.yaml
, no token is needed if SSH is already configured on the worker node/container:
deployments:
- name: my-deployment
entrypoint: flows/flow.py:flow
source:
repo: git@github.com:org/repo.git
branch: main
- For containers/Kubernetes, mount the private key and known_hosts
, or set GIT_SSH_COMMAND='ssh -i /path/to/key -o StrictHostKeyChecking=yes'
.
- GitHub App installation tokens (non-PAT)
- Treat these just like a token in Prefect. Store it in a Secret block and reference it in `source.credentials.access_token`:
variables:
GITHUB_TOKEN: "{{ prefect.blocks.secret.github_app_token }}"
deployments:
- name: my-deployment
entrypoint: flows/flow.py:flow
source:
repo: <https://github.com/org/repo>
branch: main
credentials:
access_token: "{{ variables.GITHUB_TOKEN }}"
- This avoids using a personal token and supports rotation/least privilege.
- System Git credentials (GIT_ASKPASS/credential helpers)
- If your workers already have a Git credential helper set up (e.g., GIT_ASKPASS
, gh auth
, or enterprise SSO), Prefect can just call git clone
and it will use those credentials automatically. In that case you can omit source.credentials
.
Related docs:
- prefect.yaml reference (see the source/credentials section)
A couple of notes:
- Prefect doesn’t manage ssh-agent or known_hosts
for you; that’s provided by your worker environment (local, Docker, or Kubernetes).
- If you’re currently on Prefect 2.x using the GitHub credentials block, we recommend moving to Prefect 3.x and the source
section in prefect.yaml
. In 3.x we no longer use GitHub storage blocks; instead, use .from_source('<https://github.com/owner/repo>')
or prefect deploy
with source
.
Would you like guidance for your specific runtime (local, Docker, or Kubernetes), and can you confirm if you’re on Prefect 3.x? That will help me give exact steps for SSH vs GitHub App token setup.Fugo Takefusa
09/22/2025, 3:16 AMMarvin
09/22/2025, 3:16 AMMarvin
09/22/2025, 3:16 AMgit clone
with an SSH key. Prefect just calls Git; it doesn’t manage ssh-agent or known_hosts for you.
High-level steps
- Generate an SSH key (prefer ed25519) dedicated to Prefect
- ssh-keygen -t ed25519 -C "prefect-worker" -f ~/.ssh/id_ed25519
- Add the public key as a GitHub Deploy Key on the repo (least privilege) or to a GitHub user account
- Deploy key: repo Settings → Deploy keys → Add key
- User key: Add an SSH key to your GitHub account
- Lock down permissions
- chmod 700 ~/.ssh
- chmod 600 ~/.ssh/id_ed25519
- Trust GitHub host key
- ssh-keyscan <http://github.com|github.com> >> ~/.ssh/known_hosts
- Optionally enforce strict host checking: -o StrictHostKeyChecking=yes
- GitHub host keys: GitHub’s SSH host keys
- Test from the worker environment
- ssh -T <mailto:git@github.com|git@github.com>
should say “Hi <user/org>!”
Using SSH with Prefect deployments
- Use an SSH URL in prefect.yaml
deployments:
- name: my-deployment
entrypoint: flows/flow.py:flow
source:
repo: git@github.com:org/repo.git
branch: main
- If your key isn’t the default path, set `GIT_SSH_COMMAND`:
GIT_SSH_COMMAND="ssh -i /path/to/key -o StrictHostKeyChecking=yes"
Local worker (process)
- Put the key in ~/.ssh
for the user that runs the worker
- Ensure known_hosts
contains GitHub’s keys
- Optional: run an ssh-agent and ssh-add ~/.ssh/id_ed25519
or just rely on GIT_SSH_COMMAND
Docker workers
- Best: mount the host’s ~/.ssh
read-only
docker run \
-v $HOME/.ssh:/home/prefect/.ssh:ro \
-e GIT_SSH_COMMAND="ssh -i /home/prefect/.ssh/id_ed25519 -o StrictHostKeyChecking=yes" \
prefecthq/prefect:3
- If you must bake keys into an image, use BuildKit secrets so they don’t persist in layers
# syntax=docker/dockerfile:1.4
FROM prefecthq/prefect:3
RUN mkdir -p /home/prefect/.ssh && chmod 700 /home/prefect/.ssh
RUN ssh-keyscan <http://github.com|github.com> >> /home/prefect/.ssh/known_hosts
RUN --mount=type=secret,id=sshkey,dst=/tmp/sshkey \
install -m 600 /tmp/sshkey /home/prefect/.ssh/id_ed25519
ENV GIT_SSH_COMMAND="ssh -i /home/prefect/.ssh/id_ed25519 -o StrictHostKeyChecking=yes"
Build with: docker build --secret id=sshkey,src=./id_ed25519 .
Kubernetes workers
- Create a Secret with the private key and a ConfigMap or Secret for known_hosts
apiVersion: v1
kind: Secret
metadata:
name: github-ssh
type: Opaque
data:
id_ed25519: <base64 of private key>
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ssh-known-hosts
data:
known_hosts: |
<http://github.com|github.com> ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI...
- Mount them into the Prefect worker pod and set perms (via initContainer or fsGroup)
- In your Kubernetes work pool’s base job template, add:
- volumes, volumeMounts for /home/prefect/.ssh
- an initContainer to chmod 700
the directory and chmod 600
the key
- env: GIT_SSH_COMMAND=ssh -i /home/prefect/.ssh/id_ed25519 -o StrictHostKeyChecking=yes
Notes
- Prefer Deploy Keys for specific repos to minimize blast radius
- Avoid embedding secrets in images or environment if you can mount them at runtime
- Keep StrictHostKeyChecking
on where possible
GitHub App installation tokens (non-PAT)
You can use GitHub App installation tokens instead of PATs. Key points:
- Installation tokens are short-lived (typically 1 hour). You’ll need a way to mint them just before the clone happens.
- Prefect can consume a token like any other HTTPS token:
- via source.credentials.access_token
in prefect.yaml
, orMarvin
09/22/2025, 3:16 AMvariables:
GITHUB_TOKEN: "{{ prefect.blocks.secret.github_installation_token }}"
deployments:
- name: my-deployment
entrypoint: flows/flow.py:flow
source:
repo: <https://github.com/org/repo>
branch: main
credentials:
access_token: "{{ variables.GITHUB_TOKEN }}"
- Works for testing, but you must refresh the secret before it expires
Automated token minting approaches
- Sidecar/init step mints a fresh token and exposes it to Git
- Use a small script or gh
CLI to:
1) Create a JWT signed with your GitHub App private key
2) Exchange it for an installation token via GitHub API
3) Write creds for Git to use (e.g., .netrc
or GIT_ASKPASS
script)
- Mount the App’s private key as a secret; never hardcode it
- Kubernetes: put this in an initContainer that writes .netrc
into a shared volume mounted at /home/prefect
- .netrc
example:
machine <http://github.com|github.com>
login x-access-token
password <INSTALLATION_TOKEN>
- Then git clone <https://github.com/org/repo>
will work without putting the token in the URL
- Docker/local: run a pre-start script that writes .netrc
or sets GIT_ASKPASS
to supply the token
References
- Prefect deployments/prefect.yaml: prefect.yaml reference
- GitHub App auth: Authenticating with a GitHub App
- GitHub SSH keys: Connecting to GitHub with SSH
Recommendations
- For hands-off automation, SSH deploy keys are the least fragile option (no rotation headaches).
- If you must use GitHub App tokens, plan a short-lived token minting step that runs right before the clone (initContainer/sidecar or a wrapper script).
- Never put tokens in git URLs; prefer .netrc
, credential helpers, or Prefect’s source.credentials.access_token
.
If you can share where your workers run (local process, Docker, Kubernetes), I can give you a concrete, copy-paste config for that environment.