• m

    matta

    1 year ago
    Hey, what's the standard way of storing Secrets for Prefect Core?
  • m

    matta

    1 year ago
    We've got our own cluster that we're running it on. Is there an easy for it to talk to EKS or something? Or what are Best Practices here?
  • m

    matta

    1 year ago
    Also, just wanted to say I'm super happy that I got a new gig where we're using Prefect! 🙂
  • b

    Ben Fogelson

    1 year ago
    I’m wondering about using
    apply_map
    with functions that create parameters. Here’s a minimal example:
    from prefect import Flow, Parameter, unmapped, apply_map
    
    def func(x, y):
        z = Parameter('z')
        w = x + y + z
        return w
    
    with Flow('flow') as flow:
        x = Parameter('x')
        y = Parameter('y')
        w = apply_map(func, x, y)
    
    ---------------------------------------------------------------------------
    ValueError: Parameters must be root tasks and can not have upstream dependencies.
    The error comes when
    apply_map
    tries to add edges from the root tasks of the subgraph to the argument tasks passed to
    apply_map
    . This fails when it tries to make an edge from
    x
    to
    z
    since
    z
    is a parameter. This error seems to preclude ever using
    apply_map
    with a function that creates a parameter. I’m wondering if that constraint is necessary. My intuition is that if
    apply_map
    sees a parameter that is in the subflow but not in the parent flow, it should assume that any edges from that parameter are unmapped edges that should be added to the parent flow. Does this seem right?
  • m

    Marwan Sarieddine

    1 year ago
    Hi folks, I know flow cancelling was recently introduced in prefect, but I am curious why prefect.engine.state.State.is_cancelled() wasn't also introduced ?
    m
    1 replies
    Copy to Clipboard
  • j

    Jasono

    1 year ago
    Hi, a newbie question… If the trigger for all tasks in a flow is “all successful”, and one of the upstream task fails, all of the downstream tasks are “trigger failed” (even if the upstream failed task is restarted and succeeds). This forces me to restart the trigger-failed tasks one by one. Is this the intended behavior? or am I missing something? Should I change the trigger to “all finished” in order to avoid this manual restarting of the downstream tasks when one of the upstream tasks fails at least once?
    j
    Chris White
    38 replies
    Copy to Clipboard
  • a

    abhilash.kr

    1 year ago
    Hi Folks, Did anyone receive the prefect 5k stars goodies? - https://prefect-community.slack.com/archives/CKNSX5WG3/p1601480661016400
    a
    Jeremiah
    2 replies
    Copy to Clipboard
  • psimakis

    psimakis

    1 year ago
    Hello all, Sorry in advance for this vague issue but I couldn't be more specific 😕 In my case, flows are split in different groups (different sub-folders) based on their similarity and for my convenience, I have created a shell script to register specific groups of flows each time. so far, so good. When I register a flow I have noticed that the scheduling of another flow is deregulated. Let's say I have flow
    A
    and flow
    B
    . cron for flow B is
    0 11 * * 3
    (1 event every Wednesday at 11:00am). Last time I registered flow
    A
    I noticed that two runs had been scheduled for flow
    B
    for the upcoming Wednesday (both at 11:00am). When I re-registered flow
    B
    , scheduling was fixed. Have you any idea why this happening? Thanks. python:
    3.7.3
    prefect:
    0.13.12
    OS:
    Debian GNU/Linux 10 (buster)
    psimakis
    Chris White
    4 replies
    Copy to Clipboard
  • Radek Tomsej

    Radek Tomsej

    1 year ago
    Had anybody tried to pass results/states between tasks that are in different flows? Thank you! https://prefect-community.slack.com/archives/CL09KU1K7/p1604173648410700
  • u

    uttam K

    1 year ago
    I have written mysql connection function in different file and imported in my flow it works well when i use flow.run() But when I run using prefect server UI it says module not found . I have attached code .Please help
    u
    ale
    +1
    4 replies
    Copy to Clipboard