Thread
#prefect-community
    q

    Qwame

    2 years ago
    Hello everyone, I have a task that several other tasks depend on in my prefect flow. In airflow I could do something like
    F1 >> [f2, f3, f4, f5, f6]
    What's the best way to set these dependencies in Prefect. I notice that set_downstream doesn't accept a list of tasks. Is there any efficient way to do this in Prefect? Also does the new Prefect UI mean I don't need docker to run it? Thanks
    nicholas

    nicholas

    2 years ago
    Hi @Qwame - you can set upstream or downstream tasks by passing an array to the
    upstream_tasks
    or
    downstream_tasks
    task constructor, or using the
    set_dependencies
    method like such:
    from prefect import Flow, task
     
    @task
    def F1():
        print("F1")
        return
    
    @task
    def f2():
        print("f2")
    
    @task
    def f3():
        print("f3")
    
    with Flow("F Flow") as flow:
        f1 = F1()
    
        f2(upstream_tasks=[f1])
        f3(upstream_tasks=[f1])
    or:
    from prefect import task, Flow
    
    @task
    def F1():
        print("F1")
    
    @task
    def f2():
        print("f2")
    
    @task
    def f3():
        print("f3")
        
    with Flow("F Flow") as flow:
        F1.set_dependencies(downstream_tasks=[f2, f3])
    As for Prefect Server (and Prefect UI), each of the repos can be run without Docker, however Docker (and docker-compose) are the container/service orchestration tools we've chosen to make it simple to spin everything up at once; without Docker, right now you'll need to install dependencies and run each service individually.
    q

    Qwame

    2 years ago
    Thank you very much @nicholas
    nicholas

    nicholas

    2 years ago
    Happy to help!
    q

    Qwame

    2 years ago
    It is interesting how the second example is set up
    Usually I have to do something like F1 = F1() f2 = f2() f3 = f3() Before we can do this:
    F1.set_dependencies(downstream_tasks=[f2, f3])
    I didn't know we could call the set_dependencies without doing the above first.
    nicholas

    nicholas

    2 years ago
    Both are valid; one builds the dependency graph based on the reference to the task call (which is your example), and the other dependency graph is built based on Prefect's knowledge of your intention to call a task (understood from your passing a reference to the task itself)
    q

    Qwame

    2 years ago
    I have tried set_dependencies using both the functional and imperative API and I must admit that I'm very happy with Prefect. I haven't been successful at using the UI without docker. I am unable to use docker that's why I have to install and run each service individually. I cloned the repo, and followed the instructions to build the Prefect development server. The UI showed in the browser at port 8080 but I couldn't connect my flows to the UI. I used flow.register() but it didn't work. The error was related to connection to the port. Any idea what I might be missing?
    a

    Archie35

    1 year ago
    @Qwame did you ever figure out what was wrong? I think I am having a similar connection issue
    q

    Qwame

    1 year ago
    @Archie35 nope I could not figure out what was wrong.