Georgi Milanov
01/30/2024, 12:51 PMMarvin
01/30/2024, 12:52 PMGit
block in Prefect with the necessary credentials using the Prefect UI or CLI, securely storing your personal access token.
- The Git
block should be configured with the repository URL and branch information.
3. Prefect Flow:
Write a Prefect flow that utilizes the Git
block to pull code from the repository. The flow might look something like this in broad strokes:
python
from prefect import flow
from prefect.tasks import git
@flow
def my_flow_with_private_repo():
git_task = git.Pull(
repo_url="<https://github.com/username/private-repo.git>",
branch_name="main",
git_token="${{ secrets.GITHUB_ACCESS_TOKEN }}"
)
repo_contents = git_task()
# Your logic here, reading and processing repo_contents
# Obtain access token from a secured source, e.g., Prefect Secret
access_token = "your_access_token"
if __name__ == "__main__":
my_flow_with_private_repo()
4. Managed Execution Environment:
Deploy your flow to a managed execution environment like Prefect Cloud or Prefect Orion which will handle flow scheduling, parameterization, and execution.
5. Execute the Flow:
Trigger the execution of your flow, either via the Prefect UI or Prefect's CLI or API. The flow will read from the private repo during execution.
Please note: The above code is merely illustrative. You'd have to ensure that the use of secrets is consistent with your security policy and Prefect's best practices. Keep in mind token management and the potential for dismal failure if not handled correctly. Enjoy your venture into managed execution bliss. Or don't. It's all the same to me.Nate
01/30/2024, 2:33 PMNate
01/30/2024, 2:34 PMfrom prefect import flow
from prefect.runner.storage import GitRepository
from prefect.blocks.system import Secret
my_flow = flow.from_source(
source=GitRepository(
url="<https://github.com/org/repo.git>",
credentials={"access_token": Secret.load("github-access-token")}
),
entrypoint="flows.py:my_flow",
).deploy(..., work_pool_name="managed")