Devin Flake
03/09/2022, 11:36 PMFailed to load and execute flow run: NotGitRepository()
Devin Flake
03/09/2022, 11:37 PMfrom prefect import Flow, task
from prefect.storage import Git
from prefect.run_configs import KubernetesRun
FLOW_NAME = "git_k8s"
STORAGE = Git(
repo="cgit/analytics-techops/arte-tasks",
flow_path="arte_tasks/flows/hello_world_git_k8s.py",
repo_host="<http://git.loc.adobe.net|git.loc.adobe.net>"
)
@task(log_stdout=True)
def hello_world():
text = f"hello from {FLOW_NAME}"
print(text)
return text
with Flow(
FLOW_NAME, storage=STORAGE, run_config=KubernetesRun(),
) as flow:
hw = hello_world()
Devin Flake
03/09/2022, 11:39 PMgit clone <https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>
Zanie
@property
def git_clone_url(self) -> str:
"""
Build the git url to clone
"""
if self.git_clone_url_secret_name:
return Secret(self.git_clone_url_secret_name).get() # type: ignore
if self.use_ssh:
return f"git@{self.repo_host}:{self.repo}.git"
return f"https://{self.git_token_secret}@{self.repo_host}/{self.repo}.git"
Zanie
from prefect.utilities.git import TemporaryGitRepo
with TemporaryGitRepo("<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks.git>"):
pass
Kevin Kho
Devin Flake
03/10/2022, 10:22 PMgit_clone_url_secret_name
in the flow and that made some progress, the flow was actually 'Submitted for execution' but now it fails saying:
Failed to load and execute flow run: UnboundLocalError("local variable 'head_ref' referenced before assignment")
Devin Flake
03/10/2022, 10:22 PMKevin Kho
head_ref
in the Flow?Devin Flake
03/10/2022, 10:30 PMfrom prefect import Flow, task
from prefect.storage import Git
from prefect.run_configs import KubernetesRun
FLOW_NAME = "git_k8s"
STORAGE = Git(
repo="",
flow_path="arte_tasks/flows/hello_world_git_k8s.py",
git_clone_url_secret_name="GIT_URL"
)
# GIT_URL="<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>"
# k8s env variable: "PREFECT__CONTEXT__SECRETS__GIT_URL": "<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>"
# <https://docs.prefect.io/orchestration/concepts/secrets.html>
@task(log_stdout=True)
def hello_world():
text = f"hello from {FLOW_NAME}"
print(text)
return text
with Flow(
FLOW_NAME, storage=STORAGE, run_config=KubernetesRun(),
) as flow:
hw = hello_world()
Zanie
Devin Flake
03/10/2022, 10:31 PMrepo=""
seems to be a bug that came back per https://github.com/PrefectHQ/prefect/issues/4882Zanie
Devin Flake
03/10/2022, 10:33 PMZanie
Zanie
prefect run --name <your-flow-name> --execute
to reproduce itDevin Flake
03/10/2022, 10:37 PMDevin Flake
03/10/2022, 11:18 PMUnable to import dulwich, please ensure you have installed the git extra
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/prefect/utilities/git.py", line 40, in __enter__
from dulwich.porcelain import clone
ModuleNotFoundError: No module named 'dulwich'
Zanie
prefect[git]
locallyZanie