Tim Galvin04/20/2023, 1:52 AM
based cluster. When my workflow starts a dask schedular is also started, which has an address I could connect my browser to to see a pretty comprehensive overview of the real-time compute usage. In my situation with the
cluster there are a few hurdles to get the schedular, ultimately amounting to some port forwarded ssh connections to get to the listening port of the compute node that is hosting the dashboard. So, the 'pie in the sky' question - is it feasible to integrate into the prefect UI a panel to present the dask schedular that is servicing each flow as they are running? Essentially, adding to the
to forward the schedular dashboard to a designated resource on the prefect server that can then be matched to the flow run and presented?
Jonathan Langlois04/20/2023, 1:54 AM
Ryan Peden04/20/2023, 5:53 AM
Tim Galvin04/20/2023, 6:26 AM