• Jeff Brainerd

    Jeff Brainerd

    2 years ago
    Hey 😛refect: team, a question about dask interaction. We are looking for best practices for interrupting and then restarting a flow. Our use case is that we need to sever all database connections during a database migration. We would like to avoid restarting the flow from the beginning, but we are fine with re-starting any running tasks from the beginning (in fact I don’t think we have a choice there). We are thinking of the following steps: • pause all running tasks using graphql API • scale down dask workers to 0 (we run in fargate) • scale up dask workers after the migration • unpause the tasks Would love to hear any thoughts. Thanks! 🙏
    Jeff Brainerd
    Chris White
    3 replies
    Copy to Clipboard
  • Jeremiah

    Jeremiah

    2 years ago
    We’ve got about a third of the Prefect team and @Nate Atkins chatting over in the Core Cantina right now if anyone wants to join — @Jeff Brainerd that’s a great question, let’s discuss if you can make it! https://prefect-community.slack.com/archives/C0106HZ1CMS/p1597433468091100
  • m

    Michael Reeves

    2 years ago
    I have a quick question: how can I set a flow_run_id within the call of a
    FLOW_OBJ.run()
    ? I'm currently doing the following:
    with prefect.context(flow_run_id="asdf"):
                EMAIL.run(flow_run_id="asdfasdf", name="TEST")
    neither (context, nor the variable in the kwargs of the
    run
    function) actually change the flow_run_id when its output in the logger 😞 Do I need to wrap the flow_run_id in a state variable? If so whats the best way to do this? Finally, I'm running this flow over a dask cluster so idk if my issue here is affected by this issue: https://github.com/PrefectHQ/prefect/issues/2883
    m
    nicholas
    +1
    25 replies
    Copy to Clipboard
  • p

    Philip MacMenamin

    2 years ago
    I created a project
    vtk_flow
    using the web GUI, and am attempting to register a flow:
    $ python3 -m vtk_flow.flow
    Traceback (most recent call last):
      File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/nih-3d-resource-workflows/vtk_flow/flow.py", line 82, in <module>
        f.register(project_name='vtk_flow')
      File "/prefect-env/lib64/python3.7/site-packages/prefect/core/flow.py", line 1583, in register
        no_url=no_url,
      File "/prefect-env/lib64/python3.7/site-packages/prefect/client/client.py", line 729, in register
        project_name, project_name
    ValueError: Project vtk_flow not found. Run `client.create_project("vtk_flow")` to create it.
    I thought Client was to talk to Prefect Cloud ?
    p
    nicholas
    15 replies
    Copy to Clipboard
  • h

    Hannah Amundson

    2 years ago
    Hi! I need a specific task to run if a previous task was completed + it is a certain day of the week. Is there a way to use the schedule component for filters?
    h
    nicholas
    2 replies
    Copy to Clipboard
  • kevin

    kevin

    2 years ago
    Hey guys, is it "prefect legal" to inject context into
    prefect.context
    at runtime? More concretely, can I do something like this:
    with Flow('test') as flow:
        do_stuff()
    
    with prefect.context(foo='bar'):
        flow.run()
    and if not what would be the correct way to inject context after defining a flow?
    kevin
    nicholas
    4 replies
    Copy to Clipboard
  • m

    Maxwell Dylla

    2 years ago
    💃 Just arrived! Hello Prefect world
    m
    nicholas
    2 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    2 years ago
    Hi Folks, is there a way to force the restart of a task from within the UI ? some tasks are getting stuck indefinitely in a pending state and I am trying to force the flow to attempt to run them ...
    m
    3 replies
    Copy to Clipboard
  • t

    tas

    2 years ago
    Hi All, I am looking at setting up Prefect on GCP and I did go through the medium post regarding that. I am looking for a server less solution to run the flows that will not have the overhead of setting up k8s. Something like Fargate on AWS ?
    t
    i
    +1
    4 replies
    Copy to Clipboard
  • a

    Alex Papanicolaou

    2 years ago
    Hi, I have a quick question on cached tasks: will Prefect skip parts of the graph that are cached and don’t need to be computed so it can jump to only the tasks that do need computation?
    a
    nicholas
    9 replies
    Copy to Clipboard