Alexander van Eck
08/04/2021, 1:41 PMDaskExecutor
with a custom cluster_class
the logs in Prefect UI show me;
15:36:12 INFO DaskExecutor Creating a new Dask cluster with `dask_jobqueue.remote_slurm.RemoteSLURMCluster.with_http_api_client`...
15:36:13 INFO DaskExecutor The Dask dashboard is available at <http://10.240.3.45:8787/status>
But I know for a fact that RemoteSlurmCluster (by dask-jobqueue) also has logger.debug statement.
How do I make those logger statements show up in either the flow logs (in the agent) or in the prefect logger UI?Kevin Kho
08/04/2021, 1:47 PMAlexander van Eck
08/04/2021, 1:49 PMexport PREFECT__LOGGING__EXTRA_LOGGERS="['']"
would log everything?Kevin Kho
08/04/2021, 1:55 PMRunConfig
. You just need to add the name of the module like dask-jobquue
. Also, just to be sure, you need to lower your logging level to debug with PREFECT___LOGGING___LEVEL="DEBUG"
Alexander van Eck
08/04/2021, 1:55 PM'PREFECT__LOGGING__EXTRA_LOGGERS': "['dask_jobqueue']",
'PREFECT__LOGGING__LEVEL': 'DEBUG',
in RunConfig but that doesn’t show the logging of dask_jobqueue in Prefect UI or Prefect DockerAgentKevin Kho
08/04/2021, 3:10 PMAlexander van Eck
08/04/2021, 3:25 PMChecking flow run state...
mostly) but not anything from dask_jobqueue
Kevin Kho
08/04/2021, 3:33 PMAlexander van Eck
08/04/2021, 3:55 PM__name__
Kevin Kho
08/04/2021, 3:56 PMflow.run()
?Alexander van Eck
08/04/2021, 3:59 PMAug 4 16:00:50 workflow-01 prefect[10697]: [2021-08-04 16:00:41+0000] DEBUG - dask_jobqueue.core | Starting job: 54253
Kevin Kho
08/04/2021, 4:09 PMAlexander van Eck
08/04/2021, 4:31 PM0.14.21
and the agent is 0.15.3
Kevin Kho
08/04/2021, 4:35 PMAlexander van Eck
08/04/2021, 4:51 PMKevin Kho
08/04/2021, 4:54 PMAlexander van Eck
08/05/2021, 1:35 PMChecking flow run state...
.Kevin Kho
08/05/2021, 2:44 PMAlexander van Eck
08/17/2021, 3:23 PMKevin Kho
08/17/2021, 3:26 PMINFO
logs? Do you see the filter in the UI? If it’s displaying on the agent with --show-flow-logs, it should be in the UI. Can you show me what the agent logs look like? I wanna see if it’s formatted like the Prefect logger.Alexander van Eck
08/17/2021, 3:39 PMKevin Kho
08/17/2021, 3:39 PMdask-jobqueue
is running on a different process than the flow. because of that, the CloudHandler is not getting attached to the logger, so I’m not sure dask-jobqueue
logs can easily be brought to the Prefect UI