https://prefect.io logo
Title
d

David Beck

10/19/2022, 9:39 PM
Hi once again, I wanted to see if there is a pythonic way to configure the execution environment in our CI/CD process to submit to Prefect Cloud than what is described here: https://docs.prefect.io/ui/cloud-getting-started/#configure-execution-environment I know I can use os/subprocess calls to run the commands described here, but it would be preferred to use some conventions that might already exists. I'm searching around and haven't found anything yet, so it'd be great to get pointed in the right direction
1
r

Ryan Peden

10/19/2022, 10:04 PM
This isn't Pythonic, exactly, but for the most part instead of running the Prefect CLI commands you can just set environment variables, which might be a better fit for your CI/CD pipeline. Assuming you are using Prefect Cloud, if you have an API key and know the API URL for the workspace you want to use, you can skip all the CLI commands and just set the PREFECT_API_URL and PREFECT_API_KEY environment variables.
👍 1
d

David Beck

10/19/2022, 10:16 PM
Okay that corroborates what I have recently read. I guess the only thing I'm curious about is if the environment variables are set within the same script we make our deployments, will the settings pick up on those environment changes?
reason I ask, is a test script I am running on this very premise doesn't seem to pick it up once I make a call to
prefect.settings.<ENV_VAR>.value()
if, however, I do this within the script, call for the value shows the changes:
import prefect.settings
prefect.settings.PREFECT_API_KEY = "<https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]>"
BUT that's only if I print it without the value() call so I'm concerned that's just masking it
r

Ryan Peden

10/19/2022, 10:34 PM
Prefect reads the env variables when its module initializes, so running this for example:
import os

import prefect
import prefect.settings

os.environ["PREFECT_API_KEY"] = "abc"
print(prefect.settings.PREFECT_API_KEY.value())
You get
None
because prefect was loaded before the environment variable was set. However, if you run:
import os
os.environ["PREFECT_API_KEY"] = "abc"

import prefect.settings
print(prefect.settings.PREFECT_API_KEY.value())
You get
abc
because the environment variable was set before Prefect loaded. Another option would be to have your CI/CD tooling set the environment variables before you run your Python script. Most CI/CD tools I've worked with make that easy, but I know some make it more difficult than others.
:thank-you: 1
d

David Beck

10/20/2022, 3:10 PM
Thanks for the suggestions. I think either of those work arounds should work. I'll get testing and let you know if I have any further questions