• w

    Wai Kiat Tan

    1 year ago
    hi all, if i have an upstream prefect task submitting a job to aws batch and a downstream task which executes upon the aws batch job completes. is it good to have the downstream task to keep checking whether the aws batch job completes by using boto3 sdk? or is there a more elegant way to achieve this?
    w
    Kevin Kho
    6 replies
    Copy to Clipboard
  • ciaran

    ciaran

    1 year ago
    Don't suppose anyone is running a
    prefect agent kubernetes
    in a custom Docker Image? I can get my image running on AWS Fargate with:
    command=[
        "prefect",
        "agent",
        "ecs",
        "start",
        "--agent-address",
        "http://:8080",
        "--cluster",
        cluster.cluster_arn,
        "--task-role-arn",
        ecs_task_role.role_arn,
    ],
    But on AKS it's a whole different story, I can't get it running 😭 Essentially it can't find prefect but I can run the container locally and invoke prefect
    ciaran
    Kevin Kho
    +1
    20 replies
    Copy to Clipboard
  • Zach Schumacher

    Zach Schumacher

    1 year ago
    hey guys - noticing some slow downs in prefect cloud today. Things that typically take <1 second (e.g. loading in flows to the flows view) are taking >5 in some cases. Just wanted to make you guys aware!
    Zach Schumacher
    2 replies
    Copy to Clipboard
  • Zach Schumacher

    Zach Schumacher

    1 year ago
    possibly a dumb question, but we just now are getting to the point of running things on schedules so I’m running into this. I want to schedule a flow so that it will run on a schedule in prefect cloud, however i still want it to run right away when flow.run is invoked. How can I do that? The schedule I’m using is below:
    flow.schedule = CronSchedule("30 12 * * *")
    Zach Schumacher
    Kevin Kho
    +1
    6 replies
    Copy to Clipboard
  • k

    Kathryn Klarich

    1 year ago
    I have noticed some unexpected behavior using the prefect AWSClientWait task on long running batch jobs. Sometimes the AWSClientWait task will have failed, and the schematic and run details show the task to have failed, but the flow state is hangs in the running state for hours after. This particular task is part of a mapped flow, and other mapped flows failed on the same task, yet their flow state is failed. Any idea why this might occur? The trigger rule for the task that follows the failed task is also all_successful so it shouldn't be waiting to submit another task
    k
    Kevin Kho
    32 replies
    Copy to Clipboard
  • Justin Liu

    Justin Liu

    1 year ago
    Hey all, I’m trying to set up an ECS agent to run code stored in GitHub. Right now, I’m able to run basic code, but all of the parameters set in ECSRun() seem to be ignored, like the docker image to use and the cpu/memory. I’ve seen this in a couple other threads but was unable to follow the solutions (something about overriding?) I was able to choose an image location by setting it in the Run mode in prefect cloud UI, but I had to clear the cpu and memory options or else it would give a “string not valid” error. I’m also wondering if there’s a better way to specify the image location w/out the UI?
    Justin Liu
    Kevin Kho
    40 replies
    Copy to Clipboard
  • m

    Matthew Blau

    1 year ago
    Hello all, is there a standard way of migrating Prefect Server from one server to another? We will shortly be switching servers and would like to preserve historical run data and projects
    m
    Kevin Kho
    5 replies
    Copy to Clipboard
  • Jason Prado

    Jason Prado

    1 year ago
    Is there a way to add documentation to a Parameter that shows up in the UI?
    Jason Prado
    Kevin Kho
    4 replies
    Copy to Clipboard
  • Ben Muller

    Ben Muller

    1 year ago
    Hi Prefect devs, is there a way in prefect to only increase version numbers of flows if there is a change detected? Thinking in terms of a ci/cd pipeline where all flows are registered as part of the deployment. Or is this logic something that is up to us as the users?
    Ben Muller
    t
    2 replies
    Copy to Clipboard
  • t

    Timo

    1 year ago
    Hey everyone, is it possible to break a flow if a mapped task runs on error without executing the following mapped tasks? If not, any suggestions how to solve this problem? Szenario: Executing a pipeline for a list of files (1...n) but if one runs on failure the flow should stopp immediately. I tried it with state handlers and the flow runs on error. But all instances of the mapped task get executed before the flow runs on error.
    t
    Kevin Kho
    13 replies
    Copy to Clipboard