• Miguel Angel

    Miguel Angel

    1 year ago
    Have anyone worked with a local environment for testing in a, lets say, docker-compose fashion? Suppose that you have an aws ECS cluster to execute your flows and Prefect Cloud as backend.
    Miguel Angel
    Kyle McChesney
    +1
    15 replies
    Copy to Clipboard
  • p

    Philip MacMenamin

    1 year ago
    I'm getting "Skipped (metadata unchanged)" upon registering a flow that I've updated (as in, the source code has changed, and is being ignored.
    p
    Michael Adkins
    +1
    10 replies
    Copy to Clipboard
  • Constantino Schillebeeckx

    Constantino Schillebeeckx

    1 year ago
    hi gang! what's the best practice way of registering flows that aren't contained to just one
    .py
    file? we've split out commonly shared functionality between flows into an e.g.
    utils.py
    file which is referenced in the flow. given all the storage documentation this design doesn't seem to fit into the intended use of storage.
    Constantino Schillebeeckx
    Kevin Kho
    +1
    39 replies
    Copy to Clipboard
  • David Elliott

    David Elliott

    1 year ago
    Hey all, I've been looking at upgrading my prefect version and also my distrbuted, dask & dask-kubernetes versions for our production pipeline, and just wanted to clarify some change in behaviour that I've noticed? • Previously when I ran a flow, the k8s agent would create a job which was in effect the dask scheduler, creating and retiring pods as it needed to. In my case that
    prefect-job-xxxxx
    would create 4 ephemeral dask workers (named something like
    dask-root-xxxx
    ) • Now the behaviour I'm seeing is: ◦ K8s agent creates the
    prefect-job-xxx
    ◦ In the
    prefect-job
    logs, it gives me _prefect.DaskExecutor | Creating a new Dask cluster with
    __main__.make_cluster
    .Creating scheduler pod on cluster. This may take some time._ ◦ there are then 5x
    dask-root-xxx pods
    created, where 1 of them is a dask scheduler - ie the scheduler no-longer sits within the
    prefect-job-xx
    ? Just wanted to check if this was expected/intended behaviour - I couldn't see any reference to it in the prefect release notes • In addition, (and this is more a side note that I think the prefect k8s rbac needs updating) - I've had to add 2x more rulesets to my k8s RBAC to make it work - see these docs for what's now required. Here is specifically what's changed vs the prefect docs Thanks!
    David Elliott
    Kevin Kho
    +1
    13 replies
    Copy to Clipboard
  • Billy McMonagle

    Billy McMonagle

    1 year ago
    Hi all, I'd like to add a "create project if not exists" script into my build pipeline (for git branch based work). I know I can do
    prefect create project "My Project"
    ... is it safe to run this command multiple times?
    Billy McMonagle
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Jeff Baatz

    Jeff Baatz

    1 year ago
    For some reason on Cloud, I'm noticing that most of the UI elements aren't loading properly. Is anyone else experiencing this? I've tried it across a few different browsers. It's unfortunately bad enough that I can't really use the UI at all since almost none of the text loads. I tried clearing caches and all that good stuff already. EDIT: Oh wow, you know what... Dark mode was checked, but it doesn't seem to be working well. I forgot that toggle existed.
    Jeff Baatz
    1 replies
    Copy to Clipboard
  • n

    Nadav

    1 year ago
    Hi, when running on ECSRun and setting env variables they are auto casting strings to ints which causes boto3 exception, is there a way to avoid auto casting?
    n
    Kevin Kho
    10 replies
    Copy to Clipboard
  • Aiden Price

    Aiden Price

    1 year ago
    Hi folks, if I return a dictionary from a task, is there a way to unpack that result into
    kwargs
    for the next task downstream? Something like;
    # returns {"path": "some/restapi/path", "start_time": datetime...}
    tasks = generate_tasks(first_set)
    
    # Needs a path and start_time argument
    histories = fetch_history.map(**tasks)
    But this gives me the error
    expression after ** must be a mapping with a "str" key
    Aiden Price
    Kevin Kho
    4 replies
    Copy to Clipboard
  • st dabu

    st dabu

    1 year ago
    Hi folks, was just looking at prefect example like this.. However, let's say instead of running the steps as python code, i have some cli commands.
    with Flow("NLP") as nlp_flow:
        aws s3 cp blah blah
        sometool /data/a.csv
    Is it possible to add the steps as cli commands instead ?
    st dabu
    Kevin Kho
    2 replies
    Copy to Clipboard
  • r

    Ranu Goldan

    1 year ago
    Hi everyone. I have had encountered this error for weeks, and it keeps getting often. Prefect cloud always showing 404 on Flow pages, only to certain flow, but now it's almost for all new Flow registered I took a look on the network tools, after it got 404, it keeps hitting API and gets 200 in return, until forever. How to resolve this?
    r
    nicholas
    5 replies
    Copy to Clipboard