[2022-05-31 031711+0000] INFO - prefect.CloudFlowRunner | Beginning Flow run for 'endspider_category_pipeline'

[2022-05-31 031713+0000] INFO - prefect.DaskExecutor | Creating a new Dask cluster with

<flow>.<lambda>

...

Creating scheduler pod on cluster. This may take some time.

[2022-05-31 031716+0000] ERROR - prefect.CloudFlowRunner | Unexpected error: RuntimeError('Missing dependency kubectl. Please install kubectl following the instructions for your OS. ')

Traceback (most recent call last):

File "/usr/local/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner

new_state = method(self, state, *args, **kwargs)

File "/usr/local/lib/python3.8/site-packages/prefect/engine/flow_runner.py", line 442, in get_flow_run_state

with self.check_for_cancellation(), executor.start():

File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter

return next(self.gen)

File "/usr/local/lib/python3.8/site-packages/prefect/executors/dask.py", line 243, in start

with self.cluster_class(**self.cluster_kwargs) as cluster:

File "/home/administrator/PROJECT/endlab/endspider/category_pipeline.py", line 147, in <lambda>

File "/usr/local/lib/python3.8/site-packages/dask_kubernetes/classic/kubecluster.py", line 498, in init

super().__init__(**self.kwargs)

File "/usr/local/lib/python3.8/site-packages/distributed/deploy/spec.py", line 260, in init

self.sync(self._start)

File "/usr/local/lib/python3.8/site-packages/distributed/utils.py", line 320, in sync

return sync(

File "/usr/local/lib/python3.8/site-packages/distributed/utils.py", line 387, in sync

raise exc.with_traceback(tb)

File "/usr/local/lib/python3.8/site-packages/distributed/utils.py", line 360, in f

result = yield future

File "/usr/local/lib/python3.8/site-packages/tornado/gen.py", line 762, in run

value = future.result()

File "/usr/local/lib/python3.8/site-packages/dask_kubernetes/classic/kubecluster.py", line 636, in _start

self.forwarded_dashboard_port = await port_forward_dashboard(

File "/usr/local/lib/python3.8/site-packages/dask_kubernetes/common/networking.py", line 107, in port_forward_dashboard

port = await port_forward_service(service_name, namespace, 8787)

File "/usr/local/lib/python3.8/site-packages/dask_kubernetes/common/networking.py", line 72, in port_forward_service

check_dependency("kubectl")

File "/usr/local/lib/python3.8/site-packages/dask_kubernetes/common/utils.py", line 38, in check_dependency

raise RuntimeError(

RuntimeError: Missing dependency kubectl. Please install kubectl following the instructions for your OS.

Unclosed client session

client_session: <aiohttp.client.ClientSession object at 0x7f7bbe341af0>

Unclosed client session

client_session: <aiohttp.client.ClientSession object at 0x7f7bbe341eb0>

Unclosed connector

connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f7bbe355220>, 8337662.611438069)]']

connector: <aiohttp.connector.TCPConnector object at 0x7f7bbe341850>

Unclosed connector

connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f7bbe355dc0>, 8337662.606091255)]']

connector: <aiohttp.connector.TCPConnector object at 0x7f7bbe341c40>