• j

    joshua mclellan

    1 year ago
    is there a trigger I can use to do some logging after a run is complete? I would love to write to a database when a flow finishes without having to add a task and muddle the logic of the DAG its self
    j
    Michael Adkins
    3 replies
    Copy to Clipboard
  • Maikel Penz

    Maikel Penz

    1 year ago
    Hey ! I’m running the prefect agent on AWS EKS (Kubernetes) and when firing a job (which creates a new pod on the same cluster) I get the error
    NoCredentialsError('Unable to locate credentials')
    from the job execution. In the past I used the ECS agent and I was able to pass a
    task_role_arn
    to it. However I see that both the
    KubernetesAgent
    and the run config for Kubernetes
    KubernetesRun
    don’t have a “role” parameter to inform. My next thought was that either the role attached to the
    EKS cluster
    or the
    Fargate profile
    should do the work, but as a test I gave admin to both and I still get the credentials issue. What am I missing ?
    Maikel Penz
    Kevin Kho
    +1
    14 replies
    Copy to Clipboard
  • Constantino Schillebeeckx

    Constantino Schillebeeckx

    1 year ago
    I have a task that globs for files given a file path, in my flow I call that task with something like
    f"{os.path.dirname(os.path.realpath(__file__))}/layouts"
    , however this causes the following (when executing from Prefect cloud):
    Failed to load and execute Flow's environment: NameError("name '__file__' is not defined")
    how come that global isn't available?
    Constantino Schillebeeckx
    Chris White
    12 replies
    Copy to Clipboard
  • a

    Abhas P

    1 year ago
    Hi! I am trying to run a flow with mapped tasks with a DaskExecutor on my machine, I have a couple of queries :1. will the overall run time benefit if I run the same flow on Prefect Cloud? 2. Does prefect cloud manage the spawn of a Dask cluster resource ?
    a
    1 replies
    Copy to Clipboard
  • sean williams

    sean williams

    1 year ago
    Privacy Policy Question: I'm familiarizing myself with Prefect to see if we want to use it. One thing I'm curious about is that while disabling telemetry in my config.toml, it looks like my browser is still trying to block screen recording on localhost:8080, used by logrocket. Looking at https://www.prefect.io/legal/privacy-policy/,
    The LogRocket software has a feature that allows for the “video-like” replay of a previous user session
    it looks like disabling telemetry might not disable logrocket's session recording? Am I interpreting that correctly or am I missing a config to disable it? If anything, logrocket seems more invasive than GA telemetry. Please let me know if this belongs in a different channel. Thanks
    sean williams
    nicholas
    2 replies
    Copy to Clipboard
  • Jacq Crous

    Jacq Crous

    1 year ago
    Hi all, I’m having some trouble with retrieving data from a child flow. I have a child flow register and that sits in folder A, the child flow is an ETL process that downloads and formats data from a S3 bucket. I want to retrieve the transformed data from the child flow after it ran. I am using create_flow_run to start the child flow and it seems to run, however when I use get_task_run_result to get the data from the child flow it just gets stuck. I’m not sure if I’m not passing the correct task-slug or if the problem is related to the context of the child flow changing. In principle I want to define a ETL process that I can use in other projects and just use the generated data from the child process as the data source in the new project I’m working on. Thank you in advance for the help!
    Jacq Crous
    Kevin Kho
    12 replies
    Copy to Clipboard
  • Michael

    Michael

    1 year ago
    Is it possible to know at runtime whether a Parameter has taken its default value or been overridden?
    Michael
    Greg Roche
    +1
    5 replies
    Copy to Clipboard
  • Thomas Fredriksen

    Thomas Fredriksen

    1 year ago
    message has been deleted
    Thomas Fredriksen
    Kevin Kho
    2 replies
    Copy to Clipboard
  • t

    Terawat T

    1 year ago
    Hi ! , Is there anyone successfully submitted a flow in Openshift ? I was getting the prefect agent up and running in the cluster and try to execute the flow in it, but the state got stuck into 'Submitted for execution'. The debug log in agent didn't help much.
    DEBUG:agent:Querying for ready flow runs...
    [xx] DEBUG - agent | Creating namespaced job prefect-job-c4a440b5
    DEBUG:agent:Creating namespaced job prefect-job-c4a440b5
    [xx] DEBUG - agent | Job prefect-job-c4a440b5 created
    DEBUG:agent:Job prefect-job-c4a440b5 created
    [xx] INFO - agent | Completed deployment of flow run 549ced42-7840-43d6-89da-6f1a1f9cb378
    INFO:agent:Completed deployment of flow run 549ced42-7840-43d6-89da-6f1a1f9cb378
    t
    Kevin Kho
    6 replies
    Copy to Clipboard
  • n

    Nacho Rodriguez

    1 year ago
    Hi, Today, after using our CI/CD pipeline to push some changes to our PrefectCloud account, the information of the flow runs from earlier versions have vanished. The version numbers are right, but is like the olders never existed. Is this the intended behaviour?
    n
    Kevin Kho
    9 replies
    Copy to Clipboard