https://prefect.io logo
r

Ralph Willgoss

10/02/2020, 2:55 PM
Hi I was wondering if someone can help me with a prefect logging question. I've got logs working and have configured an additional logger using config.toml 1. How do I suppress prefect logs to say WARN level and have mine at INFO level? right now the logging level controls all logs 2. I currently have a weird behaviour where I see some other logging logging statements of mine: notice the duplication - i didnt specify the formatting for the 2nd bottom log
[2020-10-02 14:32:01] INFO - (13104) task_runner.run | Task 'calculate_payoffs': finished task run for task with final state: 'Success'
INFO:prefect.TaskRunner:Task 'calculate_payoffs': finished task run for task with final state: 'Success'
d

Dylan

10/02/2020, 4:41 PM
Hi @Ralph Willgoss, https://docs.prefect.io/core/concepts/logging.html#logging-configuration According to the above, it should only affect “Prefect” loggers.
Prefect’s logger can be configured to log from stdout. If your logger is writing to there, perhaps that’s where the duplication comes from
r

Ralph Willgoss

10/02/2020, 4:45 PM
thanks guys much appreciated i cannot work out who the second logger is whats the best way to do that?
d

Dylan

10/02/2020, 4:46 PM
That’s going to be hard to determine without taking a look at your flow
Do you feel comfortable sharing?
r

Ralph Willgoss

10/02/2020, 4:46 PM
i am using the LocalDaskExecutor - would that make any difference?
d

Dylan

10/02/2020, 4:47 PM
Maybe? Dask does have an internal logger I think
But I’ve used our Dask tools and I haven’t encountered that issue
r

Ralph Willgoss

10/02/2020, 4:48 PM
ok - im going to code up a demo, and re produce the behaviour. then i can share
d

Dylan

10/02/2020, 4:48 PM
👍
r

Ralph Willgoss

10/02/2020, 4:50 PM
what is the default logger that prefect uses, its it just called "logger" as per the docs?
d

Dylan

10/02/2020, 4:51 PM
prefect.utilities.logging.get_logger(<optional name>)
Yes, that one 👍
r

Ralph Willgoss

10/02/2020, 4:52 PM
ah but if your in a task i dont think the name is optional?
d

Dylan

10/02/2020, 4:53 PM
I believe it is
r

Ralph Willgoss

10/02/2020, 4:58 PM
when you say it should only affect prefect loggers, if I configure my logger in the extra_loggers part - is it now considered a prefect logger? i want to suppress all prefect logs and just emit my logs
@Dylan this might be an answer re adding loggers "you can provide these extra loggers to have them inherit the Prefect logging config" I read this as your loggers inherit the settings. So therefore you inherit the LEVEL as well
d

Dylan

10/02/2020, 7:14 PM
Ahh yes, I believe it is
Can I ask why you’d like to turn off all of the Prefect logs?
r

Ralph Willgoss

10/04/2020, 7:34 PM
@Dylan i would like to filter out prefects INFO level logs as they are noisy and don't add any value to me. I want the ERROR, WARN etc I would like my own logger to log at INFO level. It appears as though you cannot have this type of granularity though.