https://prefect.io logo
Title
e

emre

01/21/2020, 7:54 AM
Hi everyone! I noticed that when even one of my mapped tasks fail and is scheduled for retry, a lot fo redundant logging is done. Only one task among 50 mapped tasks fail, and upon retry all 50 of these tasks generates starting task run, finished task run logs. Along with upstream and downstream mapped tasks, which also print these logs. Of course this isn’t anything breaking, just mildly annoying. Wondering if anyone has faced this issue, and has anyone come up with a fix?
j

Jenny

01/21/2020, 2:18 PM
Hi @emre - Would changing your logging level work? Or maybe customizing your logs? You can see more information for how to do that here: https://docs.prefect.io/core/tutorials/custom-logs.html
Hope that helps but if that doesn't fix it for you, let us know and we can see if there are any other ideas!
e

emre

01/21/2020, 2:23 PM
Nope, both initial logs and the duplicate, redundant logs are
INFO
level. I could make all of them disappear, but I kind of like seeing that my task has run.
j

Jenny

01/21/2020, 2:47 PM
As background, mapped tasks are dynamically generated and so Prefect doesn't know whether they should run or not until it spins up a task-runner to examine their state. As we want to visit every task to examine it's state, the level of logs is by design. Sounds like you want something between the
INFO
and
WARNING
level of logs which we don't currently have (for the above reasons we consider the log level a "feature not a bug") but if there's a call for it we might be able to make a future optimization to quiet totally unnecessary logs.