a

    alex

    2 years ago
    Hello, how can I reuse a flow? I have a function that takes in a list of items and build a flow using those items (I don't use prefect parameters currently). I want to deploy 2 versions of that flow (they will be different since the item list will be different) . I do change the name each time but it seems like they're both being considered as the same flow and replace each other.
    Jim Crist-Harif

    Jim Crist-Harif

    2 years ago
    Hi Alex, I'm not sure I understand your issue here. Can you provide some example code to show what you're doing?
    a

    alex

    2 years ago
    Here's a snippet:
    ``` demo.py
    def get_flow(
        flow_name,
        all_feeds,
    ):
        with Flow(flow_name, state_handlers=state_handlers) as flow:
            for feed in all_feeds:
                if feed.do_x:
                    x = DoXTask(
                        feed,
                        upstream_tasks=[latest],
                        task_args=dict(name=f"XTask: {feed.feed_name}"),
                    )
                    latest = x
                # build more conditional logic, use latest as upstream
        return flow
    
    name = CLI[0]
    feeds = build_feeds(name) # list of classes
    flow = get_flow(name, feeds)
    flow.register()
    ```
    I'm running
    python demo.py alpha
    and
    python demo.py bravo
    I'm expecting 2 flows alpha and bravo but instead, alpha is getting replaced by bravo (with a version increase)
    Jim Crist-Harif

    Jim Crist-Harif

    2 years ago
    Provided the output of
    get_flow
    is indeed a flow with different names for different runs, you should have two separate flows registered (flows are unique per
    (project, name)
    tuple). Are you sure your flows are being generated with different names?
    a

    alex

    2 years ago
    Yup, I just double checked. When I clicked on archived, I see
    alpha
    with version 7. The current one is
    bravo
    with version 8
    Both their names start with the same prefix though eg.
    DemoETLFlow - alpha
    could that be an issue? Also they're both registered to the same project and have different schedules
    @Jim Crist-Harif I've figured out the issue, initially, I was using theversion_group_id when deploying my code. I did remove it, but it seemed that prefect still defaulted to the same
    vgi
    for the flows, forcing the flows to overwrite each other. Clearing the prefect postgres directory and restarting prefect server resolved the issue.
    Thanks for your help!
    Jim Crist-Harif

    Jim Crist-Harif

    2 years ago
    Glad you figured it out!