Aiden Price
12/02/2020, 10:25 PMrun_config
I can get KubernetesRun()
to work with no problems at all. But I'm unsure which run_config
to use for my pre-existing Dask cluster. I presume I need the DaskExecutor
but do I use a LocalRun
with it like I used to use a LocalEnvironment
? I'm a fan of this change by the way. Thank you all.josh
12/02/2020, 10:27 PMfrom prefect.engine.executors import DaskExecutor
flow.executor = DaskExecutor(...)
flow.run_config = KubernetesRun(...)
Aiden Price
12/02/2020, 10:28 PMjosh
12/02/2020, 10:30 PMflow.environment = LocalEnvironment(
executor=DaskExecutor("address_here")
)
that executor is now broken up into its own field on the flow object. This performed similarly where the k8s agent would create a job and then that job would still execute the tasks on the dask cluster.
If you’re using a Kubernetes agent then you should use a KubernetesRun
and subsequently if you’re using a Local agent you should use a LocalRun
Aiden Price
12/02/2020, 10:35 PMBraun Reyes
12/02/2020, 11:02 PMJim Crist-Harif
12/03/2020, 1:33 AMAiden Price
12/03/2020, 1:53 AM