Channels
#prefect-community
Title
r
Ralph Willgoss
10/02/2020, 2:55 PMHi
I was wondering if someone can help me with a prefect logging question.
I've got logs working and have configured an additional logger using config.toml
1. How do I suppress prefect logs to say WARN level and have mine at INFO level? right now the logging level controls all logs
2. I currently have a weird behaviour where I see some other logging logging statements of mine: notice the duplication - i didnt specify the formatting for the 2nd bottom log
[2020-10-02 14:32:01] INFO - (13104) task_runner.run | Task 'calculate_payoffs': finished task run for task with final state: 'Success'INFO:prefect.TaskRunner:Task 'calculate_payoffs': finished task run for task with final state: 'Success'd
Dylan
10/02/2020, 4:41 PM
Hi @Ralph Willgoss,
https://docs.prefect.io/core/concepts/logging.html#logging-configuration
According to the above, it should only affect “Prefect” loggers.
Prefect’s logger can be configured to log from stdout. If your logger is writing to there, perhaps that’s where the duplication comes from
r
Ralph Willgoss
10/02/2020, 4:45 PMthanks guys much appreciated
i cannot work out who the second logger is
whats the best way to do that?
d
Dylan
10/02/2020, 4:46 PM
That’s going to be hard to determine without taking a look at your flow
Do you feel comfortable sharing?
r
Ralph Willgoss
10/02/2020, 4:46 PMi am using the LocalDaskExecutor - would that make any difference?
d
Dylan
10/02/2020, 4:47 PM
Maybe? Dask does have an internal logger I think
But I’ve used our Dask tools and I haven’t encountered that issue
r
Ralph Willgoss
10/02/2020, 4:48 PMok - im going to code up a demo, and re produce the behaviour. then i can share
d
Dylan
10/02/2020, 4:48 PM
👍
r
Ralph Willgoss
10/02/2020, 4:50 PMwhat is the default logger that prefect uses, its it just called "logger" as per the docs?
d
Dylan
10/02/2020, 4:51 PM
prefect.utilities.logging.get_logger(<optional name>)Yes, that one 👍
r
Ralph Willgoss
10/02/2020, 4:52 PMah but if your in a task i dont think the name is optional?
d
Dylan
10/02/2020, 4:53 PM
I believe it is
r
Ralph Willgoss
10/02/2020, 4:58 PMwhen you say it should only affect prefect loggers, if I configure my logger in the extra_loggers part - is it now considered a prefect logger?
i want to suppress all prefect logs and just emit my logs
@Dylan this might be an answer re adding loggers
"you can provide these extra loggers to have them inherit the Prefect logging config"
I read this as your loggers inherit the settings. So therefore you inherit the LEVEL as well
d
Dylan
10/02/2020, 7:14 PM
Ahh yes, I believe it is
Can I ask why you’d like to turn off all of the Prefect logs?
r
Ralph Willgoss
10/04/2020, 7:34 PM@Dylan i would like to filter out prefects INFO level logs as they are noisy and don't add any value to me. I want the ERROR, WARN etc
I would like my own logger to log at INFO level. It appears as though you cannot have this type of granularity though.