Michał Augoff
05/19/2023, 5:12 PMrun_deployment
as a task in 2.10.10 and seeing a few weird things in the UI (see the video for more context):
1. Not related to deployments but worth mentioning - No arrows between tasks with input-output dependencies (code in the comment). I saw this problem being mentioned a few times in the channel but there was no response
2. The details window for subflows is empty, but it works for tasks in subflows
3. The subflow run name is different from the “run_deloyment” task responsible for running that subflow, which is unintuitive (I use flow_run_name
to specify the subflow run name with run_deployment
)
4. The parent flow is missing name in the subflow view
5. The graph doesn’t change when entering subflows - not sure if this is desired
run_deployment(
name=f"run-process/deployment-1",
flow_run_name="1"
)
@task
def task_a(x: int) -> int:
get_run_logger().info(f"Running task A with {x}")
sleep(5)
return x
@task
def task_b(x: int) -> int:
get_run_logger().info(f"Running task B with {x}")
sleep(10)
return x
@task
def task_c(x: int) -> int:
get_run_logger().info(f"Running task C with {x}")
sleep(5)
return x
@flow
def run_process(x: int) -> int:
r_a = task_a(x)
r_b = task_b(r_a)
r_c = task_c(r_b)
return r_c
@flow
async def master_process(x_list: List[int]):
await asyncio.gather(
*[
run_deployment(
name=f"run-process/deployment-{x}",
flow_run_name=f"{x}"
)
for x in x_list
]
)
if __name__ == "__main__":
asyncio.run(master_process([1, 2]))
Deployment.build_from_flow(run_process, f"deployment-1", parameters={"x": 1}).apply()
Deployment.build_from_flow(run_process, f"deployment-2", parameters={"x": 2}).apply()
Zanie
05/19/2023, 5:24 PM.submit
or return real data instead of integers smaller than 255 to see the edges between your tasksMichał Augoff
05/19/2023, 5:31 PMZanie
05/19/2023, 5:33 PMMarvin
05/19/2023, 11:37 PM