https://prefect.io logo
Title
b

Bennett Lambert

03/28/2022, 2:13 PM
We've run into another issue that I was wondering if anyone could help clarify. I am running a dask cluster on an on-premise kubernetes cluster and I have the kubernetes agent running in the same namespace. The agent is connected to prefect cloud and the jobs are running on the dask cluster successfully. However, the agent never reports any runs. Is there something else that is needed in order to connect the agent with flows executed on a dask cluster?
k

Kevin Kho

03/28/2022, 2:14 PM
How are these flows runs? Were they created with a schedule?
b

Bennett Lambert

03/28/2022, 2:16 PM
I am executing the following:
from prefect import task, Flow
from prefect.executors import DaskExecutor 
from prefect.run_configs import KubernetesRun


@task
def get_value():
    return "Example!"


@task
def output_value(value):
    print(value)


with Flow("Test") as flow:
    value = get_value()
    output_value(value)

flow.run_config = KubernetesRun(labels=None)
flow.executor = DaskExecutor("IP_ADDRESS:8786")
flow.run()
k

Kevin Kho

03/28/2022, 2:19 PM
flow.run()
does not respect the
run_config
and is just meant for testing. For agent-backed runs, you need to register the Flow with Prefect Cloud/Server and then run it from the UI or schedule or by hitting the API of the backend. Those will get picked up and deployed by the agent
b

Bennett Lambert

03/28/2022, 2:20 PM
ah ok thanks for the information.