• j

    Jocelyn Boullier

    1 year ago
    Is there a way to have a task take several tasks as argument (in a list), have it run even if one of the argument fails, and have the list passed as parameter contain the exception(s) + the results ? I'm asking because I'd like to have a Slack report based on the results of several tasks, and either I'm adding an handler for each task but then I have three slack messages, or I add it at the Flow level but then I don't have (or at least I don't know how) access to the state and results of each task.
    j
    Kevin Kho
    10 replies
    Copy to Clipboard
  • j

    Joe Schmid

    1 year ago
    Hi Prefect community, for anyone attending this week's Dask Summit online conference, I'll be giving one of the keynote presentations tomorrow morning at 9am ET along with my colleague @Jie Lou on our use of Prefect & Dask for machine learning in healthcare. Prefect (and Dask) have become very key parts of our technology and have delivered great results for our business. Paid tickets are required to attend live tomorrow, but I think the talks will get posted to the Dask Youtube channel for free access after the conference: https://summit.dask.org/schedule/presentation/62/dask-prefect-for-healthcare-machine-learning-on-aws/
    j
    Kevin Kho
    +3
    18 replies
    Copy to Clipboard
  • Zach Schumacher

    Zach Schumacher

    1 year ago
    is there anyway to pass a secret into an environment variable in prefect cloud? e.g. I know i can set env variables using a template or via the KubernetesRun object, but both approaches get displayed in plain text in the UI
    Zach Schumacher
    Kevin Kho
    13 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    1 year ago
    Hi folks, this is a question of passing a custom context to a prefect flow run. More specifically about passing a custom context with a nested dictionary.
    m
    Kevin Kho
    +2
    36 replies
    Copy to Clipboard
  • Sean Harkins

    Sean Harkins

    1 year ago
    I’m experiencing an issue with unpickling when a flow is registered from inside a conda environment. Registering the flow directly from a
    virtualenv
    which contains the necessary dependencies works and the flow runs as expected. But registering the flow from a conda environment I receive this unpickling error when running.
    Sean Harkins
    Kevin Kho
    17 replies
    Copy to Clipboard
  • Damien Ramunno-Johnson

    Damien Ramunno-Johnson

    1 year ago
    I have a quick question, I was thinking about using preemptible machines. If I was using checkpointing and caching the results I guess it would just retry the flow?
    Damien Ramunno-Johnson
    Kevin Kho
    +2
    14 replies
    Copy to Clipboard
  • Jeremy Phelps

    Jeremy Phelps

    1 year ago
    Is there a way to get the agent to only run flows from a particular project?
    Jeremy Phelps
    m
    +1
    8 replies
    Copy to Clipboard
  • Peter Roelants

    Peter Roelants

    1 year ago
    Hi Prefect, Running a flow from another flow can be done through
    StartFlowRun
    . However, this needs a flow being registered prior to calling
    StartFlowRun.run()
    , which requires Prefect Cloud or server to register the flow. Is there an alternative to use dependent flows when no prefect Cloud or Server is available (for example when running a unit-test testing the full flow and its dependent flows)?
    Peter Roelants
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Mark McDonald

    Mark McDonald

    1 year ago
    Hi - I ran into a problem recently with the Task Concurrency Limiting feature. I have a task tag that is supposed to throttle the number of tasks that can interact with a particular database. In the past, this tag was only applied to a task that is mapped in a single flow. Yesterday, I tried applying this same tag to a separate mapped task in a different flow that interacts with this same database. At first, it worked as I expected. However then I saw that any task with this tag applied to it would just sit in state of "Queued" and never move into "Running". I made sure that no other tasks were running with tasks using this tag, but the behavior remained the same. I ended up having to delete the tag to get the tasks to run. My questions are:1. is it appropriate to have a single task tag shared between different flow's tasks that have a common resource dependency limitation like the one I've described? 2. Any idea why all these tasks were getting stuck in a "Queued" state when there were no actively running tasks with that tag?
    Mark McDonald
    Kevin Kho
    29 replies
    Copy to Clipboard
  • r

    Richard Hughes

    1 year ago
    Hi, I am looking for something to either run to look at long running flows and send alerts based upon these flow running over let's say an hour for example. Does anyone have a good reference to something similar or have any ideas for these types of issues I am having? Thanks in advance!
    r
    Kevin Kho
    14 replies
    Copy to Clipboard