https://prefect.io logo
m

Moe

08/02/2023, 1:01 PM
Hello guys, My prefect flows were running fine for 2 days scheduled every 2 hours. But now they are failing with no real information in logs. My work queue is healthy and my flow is deployed on ECS where the worker is running. I am using git clone step in my deployment, the logs just say
Cloning into Repo....
and it goes into pending for 8 hours or so before failing. Anyone experience this before?
j

Jake Kaplan

08/02/2023, 3:37 PM
Is it possible the github access token expired or something similar or it's waiting for input based on configuration? Do you get an error or logs of any kind?
m

Moe

08/02/2023, 4:19 PM
I don't get any error logs. My github access token is not expiring for next 11 months.
Oh, the only one that failed produced this error:
Copy code
raise RuntimeError(
RuntimeError: Failed to clone repository
That's after getting stuck for 8 hours.
The rest are all stuck at pending.
@Jake Kaplan Sorry, gitlab* token not github. But shouldn't matter.
j

Jake Kaplan

08/02/2023, 4:27 PM
do you have the full log there?
it should have an exit code on the next line
Copy code
except subprocess.CalledProcessError as exc:
        # Hide the command used to avoid leaking the access token
        exc_chain = None if access_token else exc
        raise RuntimeError(
            f"Failed to clone repository {repository!r} with exit code"
            f" {exc.returncode}."
        ) from exc_chain
m

Moe

08/02/2023, 4:28 PM
Copy code
raise RuntimeError(
RuntimeError: Failed to clone repository '<https://gitlab.com/...>' with exit code 128.