https://prefect.io
Join Slack
Prefect logging question: When I ran below from Mac, both info and debug rows (2) were displayed, bu...
# ask-community
p
Prefect logging question: When I ran below from Mac, both info and debug rows (2) were displayed, but when I registered and ran flow from Cloud, only 1 row, the logger.info row is shown, please advise logger=prefect.context.get("logger") logger.setLevel(logging.DEBUG) logger.info(‘this is test info ) logger.debug('this is test debug')
k
Did you set the logging level on the Flow?
p
No
k
You can do it on the RunConfig
Copy code
flow.run_config = RunConfig(..., env={"PREFECT__LOGGING__LEVEL": "DEBUG"}
p
Not working: I ran same flow locally and cloud and put them side by side , as you can see the 2 DEBUG messages is not showing in cloud
k
This will be hard for me to guess without seeing more code. Could you give me a small example of something I can run?
p
Copy code
from prefect import Flow, task
from prefect.executors import LocalExecutor
from prefect.run_configs import LocalRun
import prefect
import logging


def get_logger():
    logger=prefect.context.get('logger')
    logger.setLevel(logging.DEBUG)
    <http://logger.info|logger.info>('task INFO')
    logger.debug('task DEBUG')
    return logger

def get_flow():
    get_logger_task = task(get_logger)
    with Flow("logger-flow") as f:
        get_logger_task()
        f.run_config = LocalRun(labels=['qa'])
    return f

flow = get_flow()


if __name__ == "__main__":
    flow.executor = LocalExecutor()
    flow.register("default")
    flow.run()
compare output running from Cloud after registered, and ran locally
k
Thanks! Will try and explore this in a bit
But if change your flow to add the logging level I get:
Copy code
def get_flow():
    get_logger_task = task(get_logger)
    with Flow("logger-flow") as f:
        get_logger_task()
        f.run_config = LocalRun(env={"PREFECT__LOGGING__LEVEL": "DEBUG"})
    return f
So the logger is created from context and I think any changes to it will only persist inside a task. I suggest you just use the global logging level. Are you trying to do something that requires multiple logging levels?
p
I like to log detail information in debug level. In the case there is issue, I can set to debug level so I can troubleshoot.
k
I don’t think the logging configuration stops that? You can just toggle the logging level to debug when you want to see debug level logs
p
Got it, Thanks!
In the same topic. I have parent flow that run child flow using create_flow_run.map... how do I propagate same log level from parent flow to child flow?
Copy code
create_flow_run.map(
    flow_name=unmapped("livelots flow"),
    project_name=unmapped("LiveLots-ETL"),
    idempotency_key=idempotency_keys,
    parameters=filelist,
)
k
The only way would be by adding a RunConfig with an env variable. 
create_flow_run
takes in a RunConfig
p
It is working! Thanks again!
k
Of course!
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by