m

    Marwan Sarieddine

    2 years ago
    Question about setting prefect environment variables if I set the environment variable prior to running the script then it works fine
    export PREFECT__LOGGING__LEVEL="ERROR"
    but if I set the environment variable from within the script - prefect doesn’t pick up on it
    import os
    
    os.environ["PREFECT__LOGGING__LEVEL"] = "ERROR"
    
    ....
    
    flow.run()
    what am I missing here ? (is there a way to set the variable programmatically from within the script? )
    Chris White

    Chris White

    2 years ago
    Hi Marwan - you can’t update your OS environment from within a process; you should set
    PREFECT__LOGGING__LEVEL
    prior to starting your process
    m

    Marwan Sarieddine

    2 years ago
    @Chris White - thank you for the quick response - trying to use prefect while running a sagemaker training job (which unfortunately doesn’t allow passing environment variables when creating a training job … - still not sure what the workaround would be)
    Chris White

    Chris White

    2 years ago
    Interesting; the only way I could think is to run your code in another subprocess (using the
    subprocess
    or
    multiprocessing
    modules) so that you can control the environment variables. That feels very hacky to me though…
    m

    Marwan Sarieddine

    2 years ago
    Yep that’s what I am thinking as well …
    thanks !
    Chris White

    Chris White

    2 years ago
    anytime!
    emre

    emre

    2 years ago
    Another hacky solution: Change the environment variable before importing anything related to prefect.
    import os
    os.environ["PREFECT__LOGGING__LEVEL"] = "ERROR"
    import prefect
    ....
    flow.run()
    worked for me.