Jack Sundberg
09/09/2021, 3:17 PMKevin Kho
Jack Sundberg
09/09/2021, 3:23 PMJack Sundberg
09/09/2021, 3:26 PMKevin Kho
task_runner
specifically because it’s very core to Prefect. I do have an idea actually, you can explicitly create a filter for the Python logger to hide some logs. Have you seen this before?Kevin Kho
import prefect
from prefect import task, Flow
import logging
# Creating a filter
class SecureFilter(logging.Filter):
def filter(self, rec):
if 'tes' in rec.msg:
return 0
return 1
@task
def abc():
logger = prefect.context.get("logger")
logger.addFilter(SecureFilter())
<http://logger.info|logger.info>("teting") # gets shows
<http://logger.info|logger.info>("testing") # gets hidden
return 1
@task
def bcd():
logger = prefect.context.get("logger")
logger.addFilter(SecureFilter())
<http://logger.info|logger.info>("teting") # gets shows
<http://logger.info|logger.info>("testing") # gets hidden
return 1
with Flow("test") as flow:
abc()
bcd()
flow.run()
Jack Sundberg
09/09/2021, 3:33 PMJack Sundberg
09/09/2021, 3:35 PMlogger = prefect.context.get("logger")
logger.addFilter(SecureFilter())
Kevin Kho
with Flow("test") as flow:
logger = prefect.context.get("logger")
logger.addFilter(SecureFilter())
abc()
bcd()
so that it affect Parameters, but then the Flow block is evaluated during build time so I think your logger will only be configured when you register. In order for it to apply to the flow run, you need to store_as_script=True
in your storage.Kevin Kho
Jack Sundberg
09/09/2021, 3:37 PMModuleStorage
, which I believe behaves like store_as_script=True
Kevin Kho
Kevin Kho
Jack Sundberg
09/09/2021, 3:50 PM