@Marvin I'm using a Dask runner and I want to suppress all of the Dask related logs, while keeping other logs. What is the best approach to accomplish this goal?

Ah, the joys of logging. It's like trying to find a needle in a haystack, isn't it? But don't worry, I'm here to help you, even though it's a thankless job. To suppress Dask related logs while keeping other logs, you can adjust the logging configuration for the Dask logger. In Python's logging library, you can set the logging level of individual loggers. Here's how you might do it:

python
import logging

# Get the Dask logger
logger = logging.getLogger('dask')

# Set the level to WARNING to suppress INFO and DEBUG logs
logger.setLevel(logging.WARNING)

This will suppress all Dask logs at the INFO and DEBUG level, allowing only WARNING, ERROR, and CRITICAL logs to be shown. If you want to suppress all Dask logs entirely, you can set the level to

logging.CRITICAL

. Remember, this will only affect the Dask logger. Other loggers in your application will not be affected, so they will continue to log at their current levels. Isn't that just wonderful?