• Ievgenii Martynenko

    Ievgenii Martynenko

    4 months ago
    Hi again, theoretical question. If most use Docker/K8s to run dataflow on (each dataflow has own image, or each flow executes common image), why do we need Agent at all?
    Ievgenii Martynenko
    Kevin Kho
    3 replies
    Copy to Clipboard
  • a

    Adi Gandra

    4 months ago
    Hey, i’m running a prefect agent on EKS / Fargate. It’s been working great for a while now, spinning up a new pod, running and deleting it. I think for some reason something got changed on the AWS account and the spinning up of pods failed. However, it kept trying to retry and spin up a new pod, and the old pod didn’t get cleaned up. Now whenever I manually try to delete the pod (kubectl delete pods) it just recreates instantly. So now i have like 6 pods that are stuck in Pending status that I can’t delete. Any thoughts on how to clean everything up?
    a
    Kevin Kho
    +1
    18 replies
    Copy to Clipboard
  • Edmondo Porcu

    Edmondo Porcu

    4 months ago
    How would you describe Prefect in a compact way that highlights its features? It's not a workflow manager in the sense that it is not Camunda, it is not Temporal, it is not AWS Step. It's not Conductor by Netflix
    Edmondo Porcu
    Anna Geller
    5 replies
    Copy to Clipboard
  • Geoffrey Keating

    Geoffrey Keating

    4 months ago
    While
    prefect_test_harness
    provides an environment for flows to be tested, how might I go about testing a task in Prefect 2.0? It seems like calling
    .run()
    on tasks has not made it over from 1.0 yet.
    Geoffrey Keating
    Anna Geller
    +1
    6 replies
    Copy to Clipboard
  • Edmondo Porcu

    Edmondo Porcu

    4 months ago
    I am struggling a little with Prefect programming patterns. Take this example from this doc: https://docs.prefect.io/core/advanced_tutorials/task-looping.html
    with Flow("mapped-fibonacci") as mapped_flow:
        ms = Parameter("ms")
        fib_nums = compute_large_fibonacci.map(ms)
    
    flow_state = mapped_flow.run(ms=[10, 100, 1000, 1500])
    nice but what about non array parameters? Let's say I have two parameters, one that's an array, and the other which is a constant.
    with Flow("mapped-fibonacci") as mapped_flow:
        ms = Parameter("ms")
        ms2 = Parameter("ms2")
        fib_nums = compute_large_fibonacci.map(ms, ms2)
    
    flow_state = mapped_flow.run(ms=[10, 100, 1000, 1500])
    Edmondo Porcu
    Kevin Kho
    17 replies
    Copy to Clipboard
  • André Dias

    André Dias

    4 months ago
    Hi guys ! I’m trying to setup a
    Prefect Agent
    and a
    prefect server
    on my local computer, everything residing inside of
    containers
    . My
    Prefect Server
    container(s)is based on the
    docker-compose
    file retrieved from
    prefect server config > server-docker-compose.yaml
    . When spinning up a container for an Agent and running
    prefect agent local start --api <http://host.docker.internal:4200>
    , it says that I must be authenticated to communicate with the containerized server living in my local machine. My first question is: Is
    authentication
    mandatory in this kind of situation , for local run ? When I authenticate by creating a new tenant using
    prefect server create-tenant -n default
    outside the Agent container, I’m able to spin up the
    Agent Container
    and run everything. The problem is that I wanted to run the
    create-tenant
    inside the container too, but it’s not possible as I’m getting connection refused between containers.
    André Dias
    Kevin Kho
    12 replies
    Copy to Clipboard
  • Jonathan Mathews

    Jonathan Mathews

    4 months ago
    Hi, what’s the best way to switch certain flow variables depending on execution environment? e.g. a database schema of dev vs prod? (I am using set_run_config method at the moment to switch execution between local and ECS)
    Jonathan Mathews
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Patrick Tan

    Patrick Tan

    4 months ago
    How so I terminate an unknown local agent?
    Patrick Tan
    Kevin Kho
    +1
    4 replies
    Copy to Clipboard
  • Jonathan Mathews

    Jonathan Mathews

    4 months ago
    Hi! Regarding templating, I copied and pasted the code here into a new flow and it still seems to name the task “compute”, rather than “hello” Am I doing something wrong?
    Jonathan Mathews
    Kevin Kho
    2 replies
    Copy to Clipboard