Luiz Felipe
05/12/2022, 6:53 PMsleep(60)
before the end of the task, the task logs appeared, but without it it does not work.Kevin Kho
05/12/2022, 7:05 PMLuiz Felipe
05/12/2022, 7:08 PMwith Flow(
"Dask Kubernetes Flow",
storage=storage,
executor=DaskExecutor(
cluster_class=lambda: KubeCluster(make_pod_spec(image=prefect.context.image)),
adapt_kwargs={"minimum": 2, "maximum": 3},
),
run_config=KubernetesRun(),
) as flow:
Kevin Kho
05/12/2022, 7:23 PMLuiz Felipe
05/12/2022, 7:25 PMKevin Kho
05/12/2022, 7:28 PMLuiz Felipe
05/12/2022, 7:53 PMget_logs
method of KubeCluster, but given that Prefect is creating the cluster and then deleting them after running, I won't be able to run the commandKevin Kho
05/12/2022, 7:56 PMLuiz Felipe
05/12/2022, 8:45 PMtime.sleep()
on my tasks, most of the logs appeared on the Server UI?
If there is some hacky way to force them to always appear it would really work for me.Kevin Kho
05/13/2022, 6:25 PMLuiz Felipe
05/13/2022, 6:54 PM