Michał Augoff
05/19/2023, 5:12 PMrun_deployment
as a task in 2.10.10 and seeing a few weird things in the UI (see the video for more context):
1. Not related to deployments but worth mentioning - No arrows between tasks with input-output dependencies (code in the comment). I saw this problem being mentioned a few times in the channel but there was no response
2. The details window for subflows is empty, but it works for tasks in subflows
3. The subflow run name is different from the “run_deloyment” task responsible for running that subflow, which is unintuitive (I use flow_run_name
to specify the subflow run name with run_deployment
)
4. The parent flow is missing name in the subflow view
5. The graph doesn’t change when entering subflows - not sure if this is desired
run_deployment(
name=f"run-process/deployment-1",
flow_run_name="1"
)
Michał Augoff
05/19/2023, 5:13 PM@task
def task_a(x: int) -> int:
get_run_logger().info(f"Running task A with {x}")
sleep(5)
return x
@task
def task_b(x: int) -> int:
get_run_logger().info(f"Running task B with {x}")
sleep(10)
return x
@task
def task_c(x: int) -> int:
get_run_logger().info(f"Running task C with {x}")
sleep(5)
return x
@flow
def run_process(x: int) -> int:
r_a = task_a(x)
r_b = task_b(r_a)
r_c = task_c(r_b)
return r_c
Michał Augoff
05/19/2023, 5:16 PM@flow
async def master_process(x_list: List[int]):
await asyncio.gather(
*[
run_deployment(
name=f"run-process/deployment-{x}",
flow_run_name=f"{x}"
)
for x in x_list
]
)
if __name__ == "__main__":
asyncio.run(master_process([1, 2]))
Michał Augoff
05/19/2023, 5:16 PMDeployment.build_from_flow(run_process, f"deployment-1", parameters={"x": 1}).apply()
Deployment.build_from_flow(run_process, f"deployment-2", parameters={"x": 2}).apply()
Zanie
.submit
or return real data instead of integers smaller than 255 to see the edges between your tasksZanie
Michał Augoff
05/19/2023, 5:31 PMZanie
Marvin
05/19/2023, 11:37 PM