https://prefect.io
Join Slack
Is there a way to get more debug info from a flow run? I'm trying to use git storage and am getting...
# prefect-server
d
Is there a way to get more debug info from a flow run? I'm trying to use git storage and am getting this error: 
Failed to load and execute flow run: NotGitRepository()
It's a simple hello world flow:
Copy code
from prefect import Flow, task
from prefect.storage import Git
from prefect.run_configs import KubernetesRun


FLOW_NAME = "git_k8s"
STORAGE = Git(
        repo="cgit/analytics-techops/arte-tasks",
        flow_path="arte_tasks/flows/hello_world_git_k8s.py",
        repo_host="<http://git.loc.adobe.net|git.loc.adobe.net>"
)

@task(log_stdout=True)
def hello_world():
    text = f"hello from {FLOW_NAME}"
    print(text)
    return text


with Flow(
    FLOW_NAME, storage=STORAGE, run_config=KubernetesRun(),
) as flow:
    hw = hello_world()
maybe my repo or path isn't correct? from the cli the clone command works like this: 
git clone <https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>
z
Hm, we generate the clone url with
Copy code
@property
    def git_clone_url(self) -> str:
        """
        Build the git url to clone
        """
        if self.git_clone_url_secret_name:
            return Secret(self.git_clone_url_secret_name).get()  # type: ignore

        if self.use_ssh:
            return f"git@{self.repo_host}:{self.repo}.git"
        return f"https://{self.git_token_secret}@{self.repo_host}/{self.repo}.git"
You could try cloning it with the lower level utility to see if you get a more informative error, eg.
Copy code
from prefect.utilities.git import TemporaryGitRepo

with TemporaryGitRepo("<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks.git>"):
    pass
k
You can pass the Git clone url directly with this
d
Thanks @Kevin Kho and @Zanie, I tried using the prefect context secret and 
git_clone_url_secret_name
in the flow and that made some progress, the flow was actually 'Submitted for execution' but now it fails saying:
Copy code
Failed to load and execute flow run: UnboundLocalError("local variable 'head_ref' referenced before assignment")
I know it's not a very informative message, is there another way to debug that? or get more verbose output?
k
That is the error when you use an undefined variable. How did you use 
head_ref
in the Flow?
d
yeah I have no idea where that is coming from - here's what my flow looks like now:
Copy code
from prefect import Flow, task
from prefect.storage import Git
from prefect.run_configs import KubernetesRun


FLOW_NAME = "git_k8s"
STORAGE = Git(
        repo="",
        flow_path="arte_tasks/flows/hello_world_git_k8s.py",
        git_clone_url_secret_name="GIT_URL"
)
# GIT_URL="<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>"
# k8s env variable: "PREFECT__CONTEXT__SECRETS__GIT_URL": "<https://git.loc.adobe.net/cgit/analytics-techops/arte-tasks>"
# <https://docs.prefect.io/orchestration/concepts/secrets.html>

@task(log_stdout=True)
def hello_world():
    text = f"hello from {FLOW_NAME}"
    print(text)
    return text


with Flow(
    FLOW_NAME, storage=STORAGE, run_config=KubernetesRun(),
) as flow:
    hw = hello_world()
z
I suspect that this may be coming from dulwich
d
*side note, the 
repo=""
seems to be a bug that came back per https://github.com/PrefectHQ/prefect/issues/4882
z
Where is this running? There should be a full traceback in the output there
d
running from a kubernetes agent - do you know where the traceback location would be for that? I can get into the pod and check
z
It’d be the logs for the flow run pod
You could also run this flow locally (without an agent) with 
prefect run --name <your-flow-name> --execute
to reproduce it
d
I'll give that a try
looks like you were right @Zanie, from a local run:
Copy code
Unable to import dulwich, please ensure you have installed the git extra
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/prefect/utilities/git.py", line 40, in __enter__
    from dulwich.porcelain import clone
ModuleNotFoundError: No module named 'dulwich'
z
You’ll want to install 
prefect[git]
locally
We include that by default in our images so you aren’t seeing that error in K8s
7 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by