https://prefect.io logo
#prefect-community
Title
# prefect-community
m

Michael Ludwig

08/19/2020, 3:26 PM
We use the prefect logging method. Either self.logger from
Task
or imported from
from prefect.utilities.logging import get_logger
We get some weird double logging with different formats. I am not sure if it is a pure Prefect issue or just something we do wrong on our ends. But did someone else see something like this an has a solution?
Copy code
[2020-08-19 15:21:50] INFO - prefect.TaskRunner | Task 'load': finished task run for task with final state: 'Success'
Task 'load': finished task run for task with final state: 'Success'
[2020-08-19 15:21:51] INFO - prefect.TaskRunner | Task 'extract': finished task run for task with final state: 'Success'
Task 'extract': finished task run for task with final state: 'Success'
[2020-08-19 15:21:51] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
Flow run SUCCESS: all reference tasks succeeded
[2020-08-19 15:21:52] INFO - prefect.orchestrator.utils | Flow state transition: <Running: "Running flow."> -> <Success: "All reference tasks succeeded.">
Flow state transition: <Running: "Running flow."> -> <Success: "All reference tasks succeeded.">
One of the log formats can be adjusted the other one stay plain when changing this env variable:
Copy code
export PREFECT__LOGGING__FORMAT="--> [%(asctime)s] %(levelname)s - %(name)s | %(message)s"
When looking at the prefect code and playing with it I can also see that the first StreamHandler is the one giving me problems:
Copy code
def _create_logger(name: str) -> logging.Logger:
    """
    Creates a logger with a `StreamHandler` that has level and formatting
    set from `prefect.config`.

    Args:
        - name (str): Name to use for logger.

    Returns:
        - logging.Logger: a configured logging object
    """
    logging.setLogRecordFactory(_log_record_context_injector)

    logger = logging.getLogger(name)
    handler = logging.StreamHandler(sys.stdout)
    formatter = logging.Formatter(
        context.config.logging.format, context.config.logging.datefmt
    )
    formatter.converter = time.gmtime  # type: ignore
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    logger.setLevel(context.config.logging.level)

    cloud_handler = CloudHandler()
    cloud_handler.setLevel("DEBUG")
    logger.addHandler(cloud_handler)
    return logger
Nevermind we figured out that it is because of our own library we use which somehow messes with the root logger 😄