Riya Sinha
09/05/2025, 9:22 AMCode storage
You must store flow code in an accessible remote location. Prefect supports git-based cloud providers such as GitHub, Bitbucket, or GitLab. Remote block-based storage is also supported, so S3, GCS, and Azure Blob are additional code storage options.
can it be a private github repo? can i just give a key to you to access? where can i find docs on thatMarvin
09/05/2025, 9:22 AMMarvin
09/05/2025, 9:29 AMpip install "prefect[github]"
- Create and save a credentials block once (via code or the Prefect UI). For code:
from prefect_github import GitHubCredentials
GitHubCredentials(token="YOUR_FINE_GRAINED_PAT").save("gh-creds", overwrite=True)
- In your project’s prefect.yaml, add a pull step that clones your repo using that block:
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/your-org/your-repo>
branch: main
credentials: "{{ block('github-credentials', 'gh-creds') }}"
Now any Serverless run of this deployment will use the block to access your private repo.
2) Use a Prefect Variable to hold a PAT
- Define an encrypted variable in Prefect Cloud (e.g., GITHUB_TOKEN).
- Reference it in your prefect.yaml pull step:
variables:
GITHUB_TOKEN: null # set in the Prefect Cloud UI
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/your-org/your-repo>
branch: main
access_token: "{{ variable('GITHUB_TOKEN') }}"
Deploying from source
If you’re creating the deployment from code, point at your repo then edit prefect.yaml to add the pull step above:
from prefect import flow
@flow
def my_flow():
...
# Creates/updates prefect.yaml for this project
flow.from_source(
source="<https://github.com/your-org/your-repo>",
entrypoint="flows/my_flow.py:my_flow",
).deploy(
name="my-deployment",
work_pool_name="serverless", # or choose in UI
)
Then add the pull.git_clone
step with your credentials as shown above and re-run prefect deploy
from the project directory if needed.
Notes and good practices
- Use a fine‑grained GitHub PAT with minimal permissions (Contents: Read) scoped to the specific repo.
- Do not embed tokens in the repo URL (e.g., https://<token>@github.com/...). Prefer a credentials block or variable.
- SSH keys are not the recommended path for Serverless pulls; HTTPS + PAT via a credentials block or variable is simpler and supported.
- You can also use Bitbucket or GitLab via their respective credential blocks in the same way.
Verified command
- Creating/updating a deployment from a project directory:
prefect deploy --help
Docs
- Store flow code (git + auth patterns, pull steps): Store flow code
- prefect.yaml (project config, templating, variables): prefect.yaml guide
- Storing secrets/variables securely: Store secrets and variables
- GitHub integration (credentials block): prefect-github integration
- Prefect Serverless overview: Introducing Prefect Serverless
If you want, tell me your repo host (GitHub/GitLab/Bitbucket) and I can sketch the exact prefect.yaml you’d need.Jake Kaplan
09/05/2025, 2:56 PM