https://prefect.io logo
Title
m

Mdu Keswa

05/18/2021, 6:31 PM
Hello, I'm on 0.1.4.19 local Server and UI. Also have, static Dask cluster installed with helm with microK8S. Am able to successfully execute "... executor = DaskExecutor(address="<tcp://microk8s-kub-svr1:8786%7Ctcp://&lt;my-microk8s-dask-svr-ip&gt;:8786&gt;%22) flow.run(executor=executor)" on this remote Dask from python command-line. What do I need to do to 1) register the same flow with local Server (i can do this) 2) setup an agent (which one?), and 3) execute this flow from within local Server UI to run on static Dask cluster? Thanks.
k

Kevin Kho

05/18/2021, 6:37 PM
Hi @Mdu Keswa, you need to
prefect backend server
like this and then use
flow.register
instead of
flow.run
. You can also remove
flow.run
and use the CLI like
prefect register …
Check the agent types for the one you need?
and then you can either set a schedule or run immediately and it will use the DaskCluster attached to the executor in your Flow
m

Mdu Keswa

05/18/2021, 6:41 PM
Thanks Kevin! Is there an agent you can point me to - still confused which one if I'm using static Dask cluster.
n

Nitin Karolla

05/18/2021, 6:41 PM
Hi @Kevin Kho, I am working on something similar, except it compulsory to have dask cluster? Can I make it run on K8s jobs? I have registered a flow but was not really able to run it on the UI.
k

Kevin Kho

05/18/2021, 6:42 PM
The local agent might work for you @Mdu Keswa!
@Nitin Karolla, are you using Prefect to run K8s jobs?
n

Nitin Karolla

05/18/2021, 6:43 PM
I mean, I thought prefect would automatically launch k8s jobs as an executor does it not?
d

Dylan

05/18/2021, 6:44 PM
We have 3 primary concepts in our system for configuring a flow run: 1. Storage 2. Run Configs 3. Executors
The Run Config is what determines how a flow run is executed
Executors determine how task runs are executed
In your case, you’d want a KubernetesRun config to create your k8s job and either a LocalExecutor or a LocalDaskExecutor to execute task runs inside your job
LocalDaskExecutor comes with some out-of-the-box parallelism
You can use any agent/run config and connect to a static dask cluster for execution
What infrastructure do you have access to?
n

Nitin Karolla

05/18/2021, 6:59 PM
Okay, thanks for that @Dylan. I am curious, agent is the one that orchestrates everything, Would kubernetesRun still need a DaskExecutor?
d

Dylan

05/18/2021, 7:12 PM
Agents are only responsible for creating the infrastructure defined by the run config
They poll for flow runs and then create infrastructure when runs are ready to be scheduled
The executor defines how tasks execute within that infrastructure (or using adjacent/other infrastructure)
The flow run itself reports it state & the state of its tasks to the backend (Prefect Cloud or Prefect Server)
That doc is pretty comprehensive, I think it will give you a good overview
n

Nitin Karolla

05/18/2021, 7:31 PM
Thanks Dylan, this so far has been helpful
🙌 1
m

Mdu Keswa

05/18/2021, 8:49 PM
Tried with LocalAgent and Server seems to be connecting with DaskExecutor. This sample job takes seconds when submitted to static Dask cluster directly. But when ran from within UI, it doesn't seem to finish. See attached log screenshot.
k

Kevin Kho

05/18/2021, 8:53 PM
Do your tasks run successfully?
m

Mdu Keswa

05/18/2021, 9:27 PM
Hi @Kevin, it won't finish. Interesting thing is 'bokeh; view on Dask cluster for both types of execution. See attached.
Here's another 'bokeh' perspective:
k

Kevin Kho

05/18/2021, 9:32 PM
Maybe you can try this