• p

    Pierre-Edouard

    2 months ago
    Hello the community. I’m stuck in my code since days. It’s really annoying because It was supposed to be the easiest flow I develop with prefect. So, the algo: I want to read the content of a big redis database (redis.scan_iter()), filter to keep key which not matching a pattern and then remove record (easy !). My problem, I don’t want to load all my redis in memory in the first step task (my dask workers will not handle) and start filtering after. I want to generate batchs of keys and process each batch directly (map feature so). If I run my code without .map(), it work but It’s sequential (I load all my key before filtering). When I run it with .map() the filtering task return directly a failed status without any print error. I have no idea why it fail ! Someone have a clue?
    p
    redsquare
    14 replies
    Copy to Clipboard
  • Tom Klein

    Tom Klein

    2 months ago
    Hello 🙋 If we wanna use a
    LocalDaskExecutor
    for our flow, but to be able to limit the parallelisation (due to each task requiring a lot of resources when being run) to - for example - only two tasks at a time, is that possible? I read the docs but still don’t fully understand if we have to use a
    DaskExecutor
    for this, and if so - would it by default run locally (if we don’t give it any other config)? and kind of tangenial (and not directly related to prefect) but is there some advantage of something like the AWS fargate cluster for dask of a k8s dask cluster? is the former just easier to set up or something?
    Tom Klein
    p
    +1
    9 replies
    Copy to Clipboard
  • Tarek

    Tarek

    2 months ago
    hello, our cloud colleagues want to deploy prefect 2.0/orion to kubernetes, then need this information: helm chart repository and url where i could find this information?
    Tarek
    Anna Geller
    +1
    13 replies
    Copy to Clipboard
  • Andreas

    Andreas

    2 months ago
    Hi! Prefect 2.0 has the ability to enforce pydantic checks for flow input which is a really nice feature. So, when we pass invalid parameters to a flow it goes straight from a pending state to a failed state as expected and we get an error in the form of:
    09:47:51.189 | INFO    | prefect.engine - Flow run 'shiny-falcon' received invalid parameters and is marked as failed
    However this information is quite limited. Is there a way to get more information which of the parameters failed to pass the pydantic check?
    Andreas
    Anna Geller
    +1
    3 replies
    Copy to Clipboard
  • k

    Keith

    2 months ago
    Hi All! I have been loving my experience with Prefect but now that I am deploying to Kubernetes (GKE) I am having the all too common problem of it running fine locally but not in the cloud. Initially I was running into
    OOMError
    but got around that by increasing the memory requested from k8s. Now I am stuck on having the ZombieKiller stop tasks that should be reporting back status since they continue to produce logs with the message
    No heartbeat detected from the remote task; marking the run as failed.
    I have attempted to set the
    HEARTBEAT_MODE
    to
    thread
    and
    off
    via the
    config.toml
    file as well as in the
    KubernetesRun
    environment variables but no matter the combination I setup I still run into heartbeat errors killing a process. I am curious if there is a way to send a heartbeat from within the code base or if there is another approach I should take to get around my time out issue. Cheers!
    k
    Anna Geller
    +1
    8 replies
    Copy to Clipboard
  • Bogdan Serban

    Bogdan Serban

    2 months ago
    Hello everyone! I am planning to build an image processing pipeline which consists of both ML and non ML processing. I am planning to build each processing step as an independent function (Prefect Task) and successively apply these functions on each image. I will be pulling the images from a cloud storage container. The ML processing will require some GPU acceleration. My question here is twofold:1. How do I load and share the ML models to be used in running the inference? I have some pytorch models right now. 2. Is it possible to specify on what type of node (GPU/non-GPU) each task will run on? I want that the ML inference functions to run on GPU nodes, and non-ML on CPU nodes. I would really appreciate your answers! Thanks!
    Bogdan Serban
    Anna Geller
    3 replies
    Copy to Clipboard
  • m

    Marcin Grzybowski

    2 months ago
    Maybe someone will need this: So after some investigation my conclusion is that docker integration with wsl2 is the root cause. I got some successes after reinstalling wsl, but then it was failing again 😕 strange thing that it sometimes works and sometimes not. Even stranger, that when I try to run it in wsl2 docker, then it breaks wsl and I cannot even run prefect as "pure python" (without docker, just local python script) disabling wsl2 docker integration helps, but then I cannot then run docker in wsl 😞 So for now I'll stick to running it in windows docker ...
    m
    redsquare
    4 replies
    Copy to Clipboard
  • Adam

    Adam

    2 months ago
    hey folks. Hopefully simple question, I’m testing out prefect as a replacement to my org’s current tools. I want the ability to restart a failed flow manually, AFTER I make changes in the code base. Basically — I’m anticipating flows to fail and I’ll need to make tweaks to our credentials or logic in the code. But I don’t want to re-run the entire flow, just pick up at the current task where it failed after making changes. I can’t seem to register the new version of my edited flow as the same version id as the prior one. I’d need to do this because I need to restart the prior version that failed. The only thing I can do is publish the new version say (version id 15) and then version 14 that failed get’s archived and I can’t restart it with the new code base from version id 15
    Adam
    Kevin Kho
    28 replies
    Copy to Clipboard
  • j

    Jelle Vegter

    2 months ago
    Hi all, I’m wondering where most people have the prefect agent running? Do you use a virtual machine or something like Azure Container Instance? Thanks!
    j
    Anna Geller
    4 replies
    Copy to Clipboard
  • Jan Domanski

    Jan Domanski

    2 months ago
    Is prefect2 beta down for everyone or just for me?
    Jan Domanski
    Kevin Kho
    2 replies
    Copy to Clipboard