Hi I am running prefect on an HPC with HTcondor using DaskExecutor. Before the flow I start the dask cluster using dask.jobqueue I went through the documentation but it is still not clear to me if I can use the UI to monitor my processes. What i tested so far: When I start the processing by running a python script that ends with:

executor = DaskExecutor(address=cluster.scheduler_address)
……………………….
flow_state = flow.run(executor=executor)

everything runs fine and the mapped functions run in parallel. I would like to use the UI to monitor the processing and make use of the great logging functions. If I run

flow.register(project_name="test")
flow.run_agent()
flow_state = flow.run(executor=executor)

The process can start only in the UI (I guess because the run step in not executed). The process runs but run in serial and I guess because the flow.environment is not set and the default executor is used If i run

flow.environment.executor = executor
flow.register(project_name="test")
flow.run_agent()

and start the flow from the UI: the flow starts but crush with the following error:

Unexpected error: ConnectionError(MaxRetryError('None: Max retries exceeded with url: /graphql (Caused by None)'))

I guess that prefect cannot connect to the scheduler of the dask cluster. Can you please let me know if what I am trying is possible or is not implemented? If it is possible can you let me know which approach I should use? Thanks a lot! I also tested the code below as suggested in a thread but I got the same error reported above

flow.environment = RemoteDaskEnvironment(cluster.scheduler_address)
flow.register(project_name="test")
flow.run_agent()

SOLUTION I solved the issue by starting the agent outside the script.

prefect agent local start --api <http://Apollo_server_IP:4200>