https://prefect.io logo
Title
m

Marwan Sarieddine

07/19/2020, 7:36 PM
Question about setting prefect environment variables if I set the environment variable prior to running the script then it works fine
export PREFECT__LOGGING__LEVEL="ERROR"
but if I set the environment variable from within the script - prefect doesn’t pick up on it
import os

os.environ["PREFECT__LOGGING__LEVEL"] = "ERROR"

....

flow.run()
what am I missing here ? (is there a way to set the variable programmatically from within the script? )
c

Chris White

07/19/2020, 8:01 PM
Hi Marwan - you can’t update your OS environment from within a process; you should set
PREFECT__LOGGING__LEVEL
prior to starting your process
m

Marwan Sarieddine

07/19/2020, 8:04 PM
@Chris White - thank you for the quick response - trying to use prefect while running a sagemaker training job (which unfortunately doesn’t allow passing environment variables when creating a training job … - still not sure what the workaround would be)
c

Chris White

07/19/2020, 8:10 PM
Interesting; the only way I could think is to run your code in another subprocess (using the
subprocess
or
multiprocessing
modules) so that you can control the environment variables. That feels very hacky to me though…
m

Marwan Sarieddine

07/19/2020, 8:11 PM
Yep that’s what I am thinking as well …
thanks !
c

Chris White

07/19/2020, 8:11 PM
anytime!
e

emre

07/20/2020, 6:26 AM
Another hacky solution: Change the environment variable before importing anything related to prefect.
import os
os.environ["PREFECT__LOGGING__LEVEL"] = "ERROR"
import prefect
....
flow.run()
worked for me.