• Michael Law

    Michael Law

    1 year ago
    Hey folks, I'm getting a weird error on my Kubernetes cluster (AKS) when my agent kicks off its first job. The agent does start successfully, and subsequent jobs are ran to completion, so doesn't appear to stop anything functionally. Just wondered if anyone else has came across this? I am packaging up my dependencies to docker storage, and storing my flows in blob storage. Cheers
    [2021-06-23 08:53:35,179] ERROR - agent | Error while managing existing k8s jobs
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 384, in heartbeat
        self.manage_jobs()
      File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 230, in manage_jobs
        pod_events.items, key=lambda x: x.last_timestamp
    TypeError: '<' not supported between instances of 'datetime.datetime' and 'NoneType'
    ERROR:agent:Error while managing existing k8s jobs
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 384, in heartbeat
        self.manage_jobs()
      File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 230, in manage_jobs
        pod_events.items, key=lambda x: x.last_timestamp
    TypeError: '<' not supported between instances of 'datetime.datetime' and 'NoneType'
    Michael Law
    Prabin Mehta
    +1
    17 replies
    Copy to Clipboard
  • a

    Adam

    1 year ago
    Hi, I have suddenly started to get this error on my flows using the Kubernetes Agent.
    ERROR - prefect.CloudFlowRunner | Unexpected error: AttributeError("'Log' object has no attribute 'splitlines'")
    I’m running prefect self hosted. Server, agent & flow versions are 0.14.22 Nothing has changed, any idea what could be the reason?
    a
    ciaran
    +1
    20 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    1 year ago
    Hi folks, do flow concurrency limits take the flow run parameters into account ? i.e. if I don’t want two flow runs with the same parameters to run at the same time - how do I go about achieving this ?
    m
    Kevin Kho
    4 replies
    Copy to Clipboard
  • Kevin Kho

    Kevin Kho

    1 year ago
    For those not part of the #show-us-what-you-got channel.
  • s

    Sean Perry

    1 year ago
    On the Task Run views, why can’t we sort the tasks by Duration?
    s
    Kevin Kho
    10 replies
    Copy to Clipboard
  • f

    Florian Kühnlenz

    1 year ago
    Good evening from Germany, is there a difference how I activate cloud secret storage on different clients? Is there an example of how I turn in on in the config? The documentation is not very helpful about how to actually do it, for me at least.
    f
    Kevin Kho
    34 replies
    Copy to Clipboard
  • Leon Kozlowski

    Leon Kozlowski

    1 year ago
    Hi — I’m in process of trying to deploy a KubernetesAgent to EKS, I have an EKS cluster with 3 different nodes. Where would I specify which node group I would want to use for a flow run? Would it be in the flow.run_config = KubernetesRun(…)?
    Leon Kozlowski
    Kevin Kho
    +2
    9 replies
    Copy to Clipboard
  • Ben Muller

    Ben Muller

    1 year ago
    Hi are there any best practice recommendations for prefect repositories set up in production?
    Ben Muller
    Kevin Kho
    +2
    5 replies
    Copy to Clipboard
  • m

    matta

    1 year ago
    Messing with the GitHub Action again.  Does this sound right for the sequence of Actions to register flows after a Commit?1. Checkout repo into Action Runner 2. Install Prefect on the Action Runner 3. Make a config file that has a 
    config.toml
     with my auth token (stored as a secret in GitHub) 4. run 
    prefect register
     through the CLI Thanks!
    m
    Michael Adkins
    3 replies
    Copy to Clipboard
  • Ben Muller

    Ben Muller

    1 year ago
    Hey, I am using
    Docker
    storage to push an image to ECR. I am authenticated with
    aws sts assume-role
    locally in my CLI. I am trying to push the image locally at the moment and getting an error with:
    File "/Users/benmuller/code/es-betfair-flows/.venv/lib/python3.8/site-packages/prefect/storage/docker.py", line 585, in push_image
    raise InterruptedError(line.get("error"))
    InterruptedError: denied: Your authorization token has expired. Reauthenticate and try again.
    How can I make sure that my
    sts
    creds are honoured ?
    Ben Muller
    Kevin Kho
    2 replies
    Copy to Clipboard