Thread
#prefect-community
    Jack Sundberg

    Jack Sundberg

    1 year ago
    Is it possible to change the logging-level of parameter tasks? For flows that have many parameters (and most with default values), it'd be nice to remove them from the info-level logs, where they can be overwhelming.
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Jack Sundberg, I don’t see a way without setting the global setting. Also looking at the source , there’s not explicit info level logging statement, which means that it’s inheriting all of those so I think this would be hard to change unfortunately.
    Jack Sundberg

    Jack Sundberg

    1 year ago
    Okay, bummer but that's fine haha. Thanks for checking though!
    Also I just did a quick check myself -- the logger calls are actually in the task_runner class. So I'd have to open a pull request for this and just add a few if-statements
    Kevin Kho

    Kevin Kho

    1 year ago
    I honestly am not sure we’d accept PR’s into the
    task_runner
    specifically because it’s very core to Prefect. I do have an idea actually, you can explicitly create a filter for the Python logger to hide some logs. Have you seen this before?
    import prefect
    from prefect import task, Flow
    import logging
    
    # Creating a filter
    class SecureFilter(logging.Filter):
        def filter(self, rec):
            if 'tes' in rec.msg:
                return 0
            return 1
    
    @task
    def abc():
        logger = prefect.context.get("logger")
        logger.addFilter(SecureFilter())
        <http://logger.info|logger.info>("teting")        # gets shows
        <http://logger.info|logger.info>("testing")       # gets hidden
        return 1
    
    @task
    def bcd():
        logger = prefect.context.get("logger")
        logger.addFilter(SecureFilter())
        <http://logger.info|logger.info>("teting")        # gets shows
        <http://logger.info|logger.info>("testing")       # gets hidden
        return 1
    
    with Flow("test") as flow:
        abc()
        bcd()
    
    flow.run()
    Jack Sundberg

    Jack Sundberg

    1 year ago
    oh woah, thank you!
    Am I able to do these two lines outside of a task? Or the context only accessible inside a task?
    logger = prefect.context.get("logger")
    logger.addFilter(SecureFilter())
    Kevin Kho

    Kevin Kho

    1 year ago
    For you though it’s a bit trickier because you need it in the Flow block:
    with Flow("test") as flow:
        logger = prefect.context.get("logger")
        logger.addFilter(SecureFilter())
        abc()
        bcd()
    so that it affect Parameters, but then the Flow block is evaluated during build time so I think your logger will only be configured when you register. In order for it to apply to the flow run, you need to
    store_as_script=True
    in your storage.
    I believe you should have context if stored as script. How are you storing this Flow?
    Jack Sundberg

    Jack Sundberg

    1 year ago
    I use
    ModuleStorage
    , which I believe behaves like
    store_as_script=True
    Kevin Kho

    Kevin Kho

    1 year ago
    That sounds right. Let me give this a test also.
    Ok it didnt work for me. Let me ask the team if there’s another way to do this
    Jack Sundberg

    Jack Sundberg

    1 year ago
    Yeah, I'm struggling to get it working too. If there's no easy way, don't worry about it. It probably won't be worth the effort lol