https://prefect.io logo
Title
m

Michał Augoff

05/19/2023, 5:12 PM
hey team, I’m trying out
run_deployment
as a task in 2.10.10 and seeing a few weird things in the UI (see the video for more context): 1. Not related to deployments but worth mentioning - No arrows between tasks with input-output dependencies (code in the comment). I saw this problem being mentioned a few times in the channel but there was no response 2. The details window for subflows is empty, but it works for tasks in subflows 3. The subflow run name is different from the “run_deloyment” task responsible for running that subflow, which is unintuitive (I use
flow_run_name
to specify the subflow run name with
run_deployment
) 4. The parent flow is missing name in the subflow view 5. The graph doesn’t change when entering subflows - not sure if this is desired
run_deployment(
    name=f"run-process/deployment-1",
    flow_run_name="1"
)
1
This doesn’t produce arrows in any case (local run or deployment):
@task
def task_a(x: int) -> int:
    get_run_logger().info(f"Running task A with {x}")
    sleep(5)
    return x


@task
def task_b(x: int) -> int:
    get_run_logger().info(f"Running task B with {x}")
    sleep(10)
    return x


@task
def task_c(x: int) -> int:
    get_run_logger().info(f"Running task C with {x}")
    sleep(5)
    return x


@flow
def run_process(x: int) -> int:
    r_a = task_a(x)
    r_b = task_b(r_a)
    r_c = task_c(r_b)
    return r_c
Parent flow code:
@flow
async def master_process(x_list: List[int]):
    await asyncio.gather(
        *[
            run_deployment(
                name=f"run-process/deployment-{x}",
                flow_run_name=f"{x}"
            )
            for x in x_list
        ]
    )


if __name__ == "__main__":
    asyncio.run(master_process([1, 2]))
Deployments:
Deployment.build_from_flow(run_process, f"deployment-1", parameters={"x": 1}).apply()

Deployment.build_from_flow(run_process, f"deployment-2", parameters={"x": 2}).apply()
z

Zanie

05/19/2023, 5:24 PM
Use
.submit
or return real data instead of integers smaller than 255 to see the edges between your tasks
🙌 1
We can’t track some data types. Python optimizes small integers to share a memory address and they cannot be uniquely identified.
m

Michał Augoff

05/19/2023, 5:31 PM
thanks, I can confirm this works, learning something new every day 😄
z

Zanie

05/19/2023, 5:33 PM
Yeah it’s unfortunate that it means things don’t work in toy examples 😄
m

Marvin

05/19/2023, 11:37 PM
Silly @Jenny, you can't do that!
😂 2
😳 1