<@ULVA73B9P> Write python deploy script, which wil...
# ask-marvin
r
@Marvin Write python deploy script, which will deploy flow named test_flow. Flow code should be pulled from gilab repository with concrete commit_sha and using gitlab access token in credentials.
m
thought for 254.8 seconds
Here’s a complete Prefect 3.x Python deploy script that pulls the flow code from a private GitLab repo at a specific commit SHA using a GitLab access token as credentials. It deploys the
test_flow
function from your repo.
Copy code
import os
import sys
from prefect import Flow
from prefect.storage import GitRepository

def main():
    # Required configuration
    repo_url = os.environ.get("GITLAB_REPO_URL")  # e.g., "<https://gitlab.com/group/project.git>"
    commit_sha = os.environ.get("GITLAB_COMMIT_SHA")  # e.g., "a1b2c3d4..."
    gitlab_token = os.environ.get("GITLAB_TOKEN")  # GitLab personal access token
    # Path to the flow function in the repo: "<relative_path_to_py_file>:<flow_function_name>"
    # For example: "flows/my_flow.py:test_flow"
    entrypoint = os.environ.get("PREFECT_ENTRYPOINT", "flows/my_flow.py:test_flow")

    if not repo_url:
        print("ERROR: Set GITLAB_REPO_URL to your repository URL, e.g. <https://gitlab.com/group/project.git>", file=sys.stderr)
        sys.exit(1)
    if not commit_sha:
        print("ERROR: Set GITLAB_COMMIT_SHA to the commit SHA you want to deploy.", file=sys.stderr)
        sys.exit(1)
    if not gitlab_token:
        print("ERROR: Set GITLAB_TOKEN to a GitLab personal access token with repo read access.", file=sys.stderr)
        sys.exit(1)

    # Optional config
    work_pool_name = os.environ.get("PREFECT_WORK_POOL_NAME")  # e.g., "process-pool" (optional)
    include_submodules = os.environ.get("GIT_INCLUDE_SUBMODULES", "false").lower() == "true"

    # Use GitRepository with credentials to authenticate against GitLab
    repo = GitRepository(
        url=repo_url,
        commit_sha=commit_sha,
        include_submodules=include_submodules,
        # Pass token via credentials dict (avoids hardcoding into URL)
        credentials={"access_token": gitlab_token}
    )

    # Load the flow from the repo source and entrypoint
    flow = Flow.from_source(source=repo, entrypoint=entrypoint)

    # Deploy the flow. You can add schedules, tags, etc. as needed.
    deployment_id = flow.deploy(
        name="test_flow",
        work_pool_name=work_pool_name,  # can be None if you are using runner deployments
        # parameters={}, tags=[], description="...", etc.
    )

    print(f"Deployment created with ID: {deployment_id}")

if __name__ == "__main__":
    main()
Usage example: - Set env vars, then run the script:
Copy code
export GITLAB_REPO_URL="<https://gitlab.com/<group>/<project>.git>"
export GITLAB_COMMIT_SHA="<your_commit_sha>"
export GITLAB_TOKEN="<your_gitlab_access_token>"
export PREFECT_ENTRYPOINT="path/to/file.py:test_flow"
# optional
# export PREFECT_WORK_POOL_NAME="process-pool"

python deploy.py
r
@Marvin Modify script to use Gitlab credentials block.
m
thought for 157.4 seconds
Great—here’s the updated Prefect 3.x deploy script that uses a saved GitLab credentials block instead of passing the token directly. It expects: - A Prefect block of type
GitCredentials
saved in your workspace (e.g., named
gitlab-creds
) with your GitLab personal access token configured. - The repository URL, commit SHA, entrypoint, and block name provided via environment variables.
Copy code
import os
import sys
from prefect import Flow
from prefect.storage import GitRepository, GitCredentials

def main():
    # Required configuration
    repo_url = os.environ.get("GITLAB_REPO_URL")  # e.g., "<https://gitlab.com/group/project.git>"
    commit_sha = os.environ.get("GITLAB_COMMIT_SHA")  # e.g., "a1b2c3d4..."
    creds_block_name = os.environ.get("PREFECT_GIT_CREDENTIALS_BLOCK")  # e.g., "gitlab-creds"
    # Path to the flow function in the repo: "<relative_path_to_py_file>:<flow_function_name>"
    # For example: "flows/my_flow.py:test_flow"
    entrypoint = os.environ.get("PREFECT_ENTRYPOINT", "flows/my_flow.py:test_flow")

    if not repo_url:
        print("ERROR: Set GITLAB_REPO_URL to your repository URL, e.g. <https://gitlab.com/group/project.git>", file=sys.stderr)
        sys.exit(1)
    if not commit_sha:
        print("ERROR: Set GITLAB_COMMIT_SHA to the commit SHA you want to deploy.", file=sys.stderr)
        sys.exit(1)
    if not creds_block_name:
        print("ERROR: Set PREFECT_GIT_CREDENTIALS_BLOCK to the name of your saved Git credentials block (e.g., 'gitlab-creds').", file=sys.stderr)
        sys.exit(1)

    # Optional config
    work_pool_name = os.environ.get("PREFECT_WORK_POOL_NAME")  # e.g., "process-pool" (optional)
    include_submodules = os.environ.get("GIT_INCLUDE_SUBMODULES", "false").lower() == "true"

    # Load Git credentials from Prefect block
    try:
        git_creds = GitCredentials.load(creds_block_name)
    except Exception as exc:
        print(f"ERROR: Failed to load Git credentials block '{creds_block_name}': {exc}", file=sys.stderr)
        sys.exit(1)

    # Configure repository source (private GitLab via credentials block)
    repo = GitRepository(
        url=repo_url,
        commit_sha=commit_sha,
        include_submodules=include_submodules,
        credentials=git_creds,
    )

    # Load the flow from the repo source and entrypoint
    flow = Flow.from_source(source=repo, entrypoint=entrypoint)

    # Deploy the flow
    deployment_id = flow.deploy(
        name="test_flow",
        work_pool_name=work_pool_name,  # optional; can be None if using runner deployments
        # parameters={}, tags=[], description="...", etc.
    )

    print(f"Deployment created with ID: {deployment_id}")

if __name__ == "__main__":
    main()
How to create the Git credentials block (one-time setup): - In code (example):
Copy code
from prefect.storage import GitCredentials

GitCredentials(access_token="<your_gitlab_access_token>").save("gitlab-creds", overwrite=True)
- Or create it in the Prefect UI as a “Git Credentials” block, set the access token, and save as “gitlab-creds”. Run the script:
Copy code
export GITLAB_REPO_URL="<https://gitlab.com/<group>/<project>.git>"
export GITLAB_COMMIT_SHA="<your_commit_sha>"
export PREFECT_GIT_CREDENTIALS_BLOCK="gitlab-creds"
export PREFECT_ENTRYPOINT="path/to/file.py:test_flow"
# optional
# export PREFECT_WORK_POOL_NAME="process-pool"
# export GIT_INCLUDE_SUBMODULES=true

python deploy.py
Notes: - This uses generic
GitCredentials
with an access token; it works for GitLab over HTTPS. - Ensure the token has read access to the repository.