Channels
#prefect-community
Title
p
Patrick Tan
04/08/2022, 6:48 PMPrefect logging question: When I ran below from Mac, both info and debug rows (2) were displayed, but when I registered and ran flow from Cloud, only 1 row, the logger.info row is shown, please advise
logger=prefect.context.get("logger")
logger.setLevel(logging.DEBUG)
logger.info(‘this is test info )
logger.debug('this is test debug')
k
Kevin Kho
04/08/2022, 6:49 PM
Did you set the logging level on the Flow?
p
Patrick Tan
04/08/2022, 6:50 PMNo
k
Kevin Kho
04/08/2022, 6:52 PM
You can do it on the RunConfig
Copy code
flow.run_config = RunConfig(..., env={"PREFECT__LOGGING__LEVEL": "DEBUG"}p
Patrick Tan
04/08/2022, 7:14 PMNot working: I ran same flow locally and cloud and put them side by side , as you can see the 2 DEBUG messages is not showing in cloud
k
Kevin Kho
04/08/2022, 7:20 PM
This will be hard for me to guess without seeing more code. Could you give me a small example of something I can run?
p
Patrick Tan
04/08/2022, 7:41 PM Copy code
from prefect import Flow, task
from prefect.executors import LocalExecutor
from prefect.run_configs import LocalRun
import prefect
import logging
def get_logger():
logger=prefect.context.get('logger')
logger.setLevel(logging.DEBUG)
<http://logger.info|logger.info>('task INFO')
logger.debug('task DEBUG')
return logger
def get_flow():
get_logger_task = task(get_logger)
with Flow("logger-flow") as f:
get_logger_task()
f.run_config = LocalRun(labels=['qa'])
return f
flow = get_flow()
if __name__ == "__main__":
flow.executor = LocalExecutor()
flow.register("default")
flow.run()k
Kevin Kho
04/08/2022, 7:56 PM
Thanks! Will try and explore this in a bit
But if change your flow to add the logging level I get:
Copy code
def get_flow():
get_logger_task = task(get_logger)
with Flow("logger-flow") as f:
get_logger_task()
f.run_config = LocalRun(env={"PREFECT__LOGGING__LEVEL": "DEBUG"})
return fSo the logger is created from context and I think any changes to it will only persist inside a task. I suggest you just use the global logging level. Are you trying to do something that requires multiple logging levels?
p
Patrick Tan
04/11/2022, 12:42 PMI like to log detail information in debug level. In the case there is issue, I can set to debug level so I can troubleshoot.
k
Kevin Kho
04/11/2022, 1:59 PM
I don’t think the logging configuration stops that? You can just toggle the logging level to debug when you want to see debug level logs
p
Patrick Tan
04/11/2022, 5:13 PMGot it, Thanks!
In the same topic. I have parent flow that run child flow using create_flow_run.map... how do I propagate same log level from parent flow to child flow?
Copy code
create_flow_run.map(
flow_name=unmapped("livelots flow"),
project_name=unmapped("LiveLots-ETL"),
idempotency_key=idempotency_keys,
parameters=filelist,
)k
Kevin Kho
04/11/2022, 6:09 PM
The only way would be by adding a RunConfig with an env variable.
create_flow_runp
Patrick Tan
04/11/2022, 6:18 PMIt is working! Thanks again!
k
Kevin Kho
04/11/2022, 6:37 PM
Of course!
4 Views