• m

    Marwan Sarieddine

    1 year ago
    Hi folks, I am trying to figure out if there is a simpler way than what I have in mind to ensure that only one flow run for a particular flow can run at a specific point in time
    m
    Kyle Moon-Wright
    +1
    8 replies
    Copy to Clipboard
  • c

    Charles Liu

    1 year ago
    Just seeking some clarification: I was digging through old questions and came across this solution: [link: https://prefect-community.slack.com/archives/CL09KU1K7/p1615218708404000?thread_ts=1615022331.342300&cid=CL09KU1K7 ], and it says:
    # You'd set a secret in Prefect cloud containing your credentials as a JSON dictionary of the following form:
    {"ACCESS_KEY": "your aws_access_key_id here",
     "SECRET_ACCESS_KEY": "your aws_secret_access_key here"}
    
    # For now the secret name is hardcoded, so you'd need to name the secret "AWS_CREDENTIALS"
    
    # you'd then attach the secret to your `S3` storage as follows:
    flow.storage = S3(..., secrets=["AWS_CREDENTIALS"])
    Assuming AWS_CREDENTIALS has been declared on the Prefect Cloud side, is this how we pass functions into both storage=storage_type(secrets=['']) and KubernetesRun(image_pull_secrets=['']) for now?
    c
    Kyle Moon-Wright
    4 replies
    Copy to Clipboard
  • Pedro Machado

    Pedro Machado

    1 year ago
    Hi there. One of my clients is considering Prefect and someone in their company brought up (in the context of using open source tools) that, at some point, Oracle was planning to charge some licensing fees for pieces of java that were previously open source. They asked me whether Prefect uses components that may rely on java and could be negativelly impacted if there was a change in java licensing. My understand is that the code is all written in Python, but I thought I'd ask anyway. Thanks
    Pedro Machado
    Dylan
    2 replies
    Copy to Clipboard
  • j

    Justin Chavez

    1 year ago
    Hi all, when using
    StartFlowRun
    is it possible to see the logs for each called flow? I am locally running a parent flow that calls multiple child flows using
    StartFlowRun
    , but I can only see the success or failed status, not any of stdouts that some of the child flow tasks prints.
    j
    Kyle Moon-Wright
    2 replies
    Copy to Clipboard
  • j

    Julio Venegas

    1 year ago
    Hi everyone! I'm having issues getting logs displayed in UI (prefect server, local agent) from a local package that has it's own logging setup i.e. I'm not setting up logs at task level, I want to use the module's own defined logs. I had a look at the different threads after searching for "extra loggers", I'm following the steps already mentioned in several threads and in the docs. I'm attaching the flow and the package code for reference. The package is
    adls_utils
    , I'm using a
    logging.conf
    file that gets read in the main module
    dataproject.py
    . The flow is
    adls_create_day_dirs.py
    . I start the local agent with the
    local_agent_start.txt
    to set environment variables, i.e. that's where I set
    PREFECT__LOGGING__EXTRA_LOGGER
    . @Jim Crist-Harif saw you've helped other people with this topic before, hence the mention but feel free to leave it for the community to answer 😃
    j
    Kyle Moon-Wright
    6 replies
    Copy to Clipboard
  • d

    daniel

    1 year ago
    Does anybody else have issues with actually installing perfect? Running pip install perfectonly gets me this decorator
    d
    Kyle Moon-Wright
    5 replies
    Copy to Clipboard
  • asm

    asm

    1 year ago
    howdy! I tried to reset my password but it doesn't seem to have taken, and when I try and request a reset it directs me to this page: https://universal.prefect.io/help/login
    asm
    nicholas
    2 replies
    Copy to Clipboard
  • asm

    asm

    1 year ago
    wondering how I can request a password reset now?
  • c

    Charles Liu

    1 year ago
    Has anyone else had funky k8/fargate scaling behavior with Prefect? Would love some second opinions. Edit: For some context, with an S3/K8 config, I can reach slightly above 30ish% utilization on my instance, and my cluster has a fargate profile configured for it. Although I'm hitting somewhere in the low 30's, there are somehow tasks that fail outright with a heartbeat error, and presumably when I was running Docker/K8 which manifested as an HTTP error. These tasks will intermittently succeed, which is why I'm a little confused, but will otherwise fail as I'm trying to load test our instance presumably due to lack of adequate resources. On the frontend, I'm running six tasks scheduled on a 1 min interval. 4 out of the 6 tasks run just fine consistently. The two that crash are variants of one of our heavier flows, and alternate between failure and success (more failures).
  • Seonghwan Hong

    Seonghwan Hong

    1 year ago
    I use aws ecr for prefect docker storage. I log in to ECR periodically with the following command "`aws ecr get-login-password --region yourregion | docker login --username AWS --password-stdin yourregistryid.dkr.ecr.yourregion.amazonaws.com`" because the ECR token is expired after 12 hours. But In prefect agent, docker client is not updated when the token is expired. So every 12 hours, I have got 404 error and have to restart my agent.
    Seonghwan Hong
    CA Lee
    2 replies
    Copy to Clipboard