https://prefect.io logo
Title
m

Mark Williams

02/10/2020, 5:38 PM
Is there an example of how to setup a custom configuration file for Windows? It points to ~/Users/<username>. This is not a valid path for Windows users. I have tried setting the environment variables on load with no luck. I also set them in the windows environment variables and still no luck.
c

Chris White

02/10/2020, 6:13 PM
Hi @Mark Williams - what environment variable did you set?
m

Mark Williams

02/10/2020, 6:52 PM
This one: PREFECT__USER_CONFIG_PATH. I tried setting it in Windows and at runtime. Did not work either way.
c

Chris White

02/10/2020, 6:54 PM
Ok gotcha - could you open a GitHub issue for this? Definitely sounds like a bug
m

Mark Williams

02/10/2020, 7:26 PM
Before I do, this is an example of what I have (this way you could tell me if I am doing this incorrectly): TOML file:
intel_server = "${environments.${environment}}"
admin_server = "${environments.${environment}}"

[environments]
    [environments.dev]
        intel_server = "dev_intel"
        admin_server = "dev_admin"
    [environments.prod]
        intel_server = "prod_intel"
        admin_server = "prod_admin"
Python Script:
import os
import prefect

prefect.config.environment = 'dev'

os.environ["PREFECT__USER_CONFIG_PATH"] = r'C:\\Users\mark.williams\Desktop\RDW\Python\ETL\prefect_config.toml'

assert prefect.config.intel_server == 'dev_intel'
c

Chris White

02/10/2020, 8:06 PM
Ah! Yes, setting environment variables via
os.environ
generally won’t have any effect unless you then spawn a subprocess / fork. Instead I’d recommend setting that environment variable prior to creating your Python session
m

Mark Williams

02/10/2020, 8:24 PM
is there a way to set it at runtime though? There will be multiple collaborators on this project. Some that do not have access to update environment variables other than this way.
c

Chris White

02/10/2020, 8:27 PM
As long as it’s set prior to starting Python, e.g.,
PREFECT__USER_CONFIG_PATH="C:\\Path" python
m

Mark Williams

02/10/2020, 10:08 PM
Thanks @Chris White! I can setup the IDE to run it this way for testing purposes and on the production server set up the environment variable properly. Last question: Any chance you all are working on a Task Library for on premise SQL Server?
c

Chris White

02/10/2020, 10:15 PM
We don’t have anything in the pipeline for that but if you open an issue requesting it we will definitely look into it!
m

Mark Williams

02/10/2020, 10:24 PM
Great! I will also take a look and see if I can help add it to Core.
c

Chris White

02/10/2020, 10:42 PM
That would be awesome!