https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I am looking at the Logging documenta...
# marvin-in-the-wild
g
@Marvin I am looking at the Logging documentation here: https://docs.prefect.io/latest/guides/logs/ but it is not clear to me how we can configure our logs to still go to the console but not get sent to Prefect cloud. Some of the language for example around 
PREFECT_LOGGING_EXTRA_LOGGERS
says "By default, Prefect won't capture log statements from libraries that your flows and tasks use" but it is unclear if "capture" means capture by the logger at all or whether those logs get sent to Prefect. Furthermore, I do not want any Flow or Task logs to be sent to Prefect but still want them to end up on the container's stdout so that they will get picked up by fluentbit (our log processor)
m
It appears that the Prefect documentation and GitHub issues do not provide a straightforward answer to your query about configuring logs to console without sending them to Prefect Cloud, and how 
PREFECT_LOGGING_EXTRA_LOGGERS
affects this process. The 
PREFECT_LOGGING_EXTRA_LOGGERS
environment variable is mentioned in the context of which loggers are captured by Prefect by default. However, the exact mechanism for ensuring that logs are not sent to Prefect Cloud and instead only output to standard output (stdout) is not detailed in the excerpts provided. Since the direct documentation and issues search didn't yield the necessary specifics, I recommend exploring the Prefect community or support channels for more in-depth guidance. Configuring logging to ensure compliance with external log processors like fluentbit while avoiding cloud logging in Prefect might require a more nuanced approach or hands-on assistance from the Prefect team. To get a more tailored answer, you may consider either filing a new GitHub issue in the Prefect repository to clarify this configuration or reaching out on the Prefect Slack community for real-time assistance. Here are the links to begin: - Prefect GitHub Issues: Create a new issue - Prefect Slack Community: Join the Slack Workspace If you need further assistance or have other questions, feel free to ask.
n
hi @Garrett Scholtes - short answer, use 
get_logger
instead of 
get_run_logger
g
Thank you
Actually can I just not use 
APILogHandler
as a handler? Like I notice that we have this bit in our logging.yml
Copy code
handlers:
    ...
    api:
        level: 0
        class: prefect.logging.handlers.APILogHandler
and some loggers that use the api setting:
Copy code
loggers:
    prefect:
        level: "${PREFECT_LOGGING_LEVEL}"

    prefect.extra:
        level: "${PREFECT_LOGGING_LEVEL}"
        handlers: [api]

    prefect.flow_runs:
        level: NOTSET
        handlers: [api]

    prefect.task_runs:
        level: NOTSET
        handlers: [api]
(a lot of this is from the default/sample logging.yml IIRC) So instead we might be able to just set a different handler for these loggers, and possibly also configure the api handler to not use 
prefect.logging.handlers.APILogHandler
?
If I'm interpreting this and this correctly
update: I figured this out. The piece of info I was missing is that I overlooked this line:
See the Python Logging configuration documentation for more information about the configuration options and syntax used by 
logging.yml
.
Python config docs was all I needed
5 Views
Open in Slack
PreviousNext
Powered by