• c

    Christian

    2 years ago
    Hi all. I just spotted that we now have a first implementation of GreatExpectations in GitHub HEAD! Great stuff... I try to run the example and wonder how to access the GE return json with the validation results?
    c
    Jim Crist-Harif
    11 replies
    Copy to Clipboard
  • asm

    asm

    2 years ago
    is this the appropriate venue to report a bug? or is there somewhere better?
    asm
    Chris White
    4 replies
    Copy to Clipboard
  • m

    Matthias

    2 years ago
    Hi, i am running my flows on a RemoteDask Environment. I can monitor how the memory usage accumulates over time (with every scheduled run) until at one point the memory is full, dask becomes unresponsive and logs this:
    distributed.worker - WARNING - Memory use is high but worker has no data to store to disk.  Perhaps some other process is leaking memory?  Process memory: 3.43 GB -- Worker memory limit: 4.18 GB
    All the runs complete successfully and I have the feeling, but still the memory used does not get released. I am not really sure where to start debugging. Is there a way to force a memory release? The only other option I currently see is to force a dask-worker restart after the flow run finishes, but that feels very hacky.
    m
    Jim Crist-Harif
    3 replies
    Copy to Clipboard
  • Matias Godoy

    Matias Godoy

    2 years ago
    Hi guys! I've been looking around but I did not find anything: Is there a recommended way to upgrade the Prefect Core Server to a new version while in production? What I think the steps are:1. Stop the prefect server 2. Run
    pip install prefect --upgrade
    3. Run
    prefect server start
    Is this correct? Would that be it or am I missing some steps?
    Matias Godoy
    Jeremiah
    2 replies
    Copy to Clipboard
  • t

    Thomas Hoeck

    2 years ago
    Hi. I have bit of trouble getting the UI to work as it can't connect to the graphql. It keeps saying "Attempting to connect...http://localhost:4200/graphql/". I have set the host I want to use in my config.toml: ############## config.toml ############## backend = "server" [server] host = "http://srvdocker01" port = "4200" host_port = "4200" endpoint = "${server.host}😒{server.port}" [server.ui] host = "http://srvdocker01" port = "8081" host_port = "8081" endpoint = "${server.ui.host}😒{server.ui.port}" graphql_url = "http://srvdocker01:4200/graphql" ########################################## This setting is reflected when I run the server (see pic):
    t
    Jeremiah
    +1
    8 replies
    Copy to Clipboard
  • z

    Zach

    2 years ago
    I am trying to use the Orchestration GraphQL API Python Client and it seems like there is barely any documentation on it. I used the interactive API on the prefect cloud website to build my query, but it has "where" statements, and the small snippet of code in the prefect docs about the python client don't give any details on how to use "where" statements with the python client.https://docs.prefect.io/orchestration/concepts/api.html#getting-started
    z
    Kyle Moon-Wright
    +1
    12 replies
    Copy to Clipboard
  • b

    Brett Naul

    2 years ago
    is there an ETA on the 0.11.6 release? 🙂
    b
    Kyle Moon-Wright
    +1
    4 replies
    Copy to Clipboard
  • g

    goodsonr

    2 years ago
    Hi All ... prefect newbie here. Sorry if the answer to my question is obvious and somewhere. But I couldn't find it. I am struggling to figure out how to access the state of a task (task a) within another task (task b). I want task b to always run whether or not task a succeeds or fails. I want task b to do-stuff if task A fails, or skip if task A succeeds.  Sort of like this
    @task 
    def a ():
       < do something that might succeed or fail >
    @task (trigger = always_run)
    def b():
      < if task a status==FAIL .. do something >
      < if task b status=SUCCESS .. skip >    
    
    with Flow as flow:
        res1 = a()
        b(res1)
    flow.run()
    This is part of a larger flow with other tasks ahead and after a&b. I tried
    trigger=any_failed
    on task b, but that causes task b to fail if task a succeeds (due to trigger not satisfied) .. which is not what I want. I want task b to always show success. Again .. sorry if this is obvious and its just a newbie-thing. Feel free to just point me to the right place in the doc. Thanks in advance
    g
    Zachary Hughes
    +1
    11 replies
    Copy to Clipboard
  • Darragh

    Darragh

    2 years ago
    Back again, looking for advice on secrets. I’m trying to inject secrets into Fargate containers and not getting very far. As I understand it, the native AWS Secret Manager support allows me to specify a secret ARN, and if my secret name is something like CREDS, I add the following to the FargateAgent config:
    "secrets":[{"name": "CREDS", "valueFrom": "arn:aws:secretsmanager:eu-west-1:11111111:secret:local/aws/credentials-abcd"}]
    In Flow, I read like this:
    creds = prefect.context.secrets.CREDS
    But I keep getting the following:
    AttributeError: 'dict' object has no attribute 'CREDS'
    Confused face.
    Darragh
    Chris White
    17 replies
    Copy to Clipboard