@Marvin Is there a way to load a block and export the values as environment variables?

It looks like there are ways to add environment variables to your deployments using Prefect. Here's a summary: 1. Using Deployment Configuration: You can add environment variables directly in the

job_variables

section of your deployment definition in the

prefect.yaml

file. Example:

deployments:
   - name: demo-deployment
     entrypoint: demo_project/demo_flow.py:some_work
     work_pool:
       name: local
     job_variables:
       env:
         EXECUTION_ENVIRONMENT: staging
         MY_NOT_SO_SECRET_CONFIG: plumbus
     schedule: null

More details can be found here: Overriding Job Variables. 2. Using Prefect Blocks: You can also define environment variables as part of a block configuration, especially when dealing with infrastructure like ECS. Example with Kubernetes JS:

python
   from prefect.infrastructure import KubernetesJob

   k8s = KubernetesJob(
       image="prefecthq/prefect:2-python3.9",
       namespace="prod",
       image_pull_policy="IfNotPresent",
       env={"PREFECT_LOGGING_LEVEL": "DEBUG", "EXTRA_PIP_PACKAGES": "s3fs"},
   )
   k8s.save("prod", overwrite=True)

You can then modify the environment variables dynamically:

python
   new = KubernetesJob.load("prod")
   new.env["PREFECT_LOGGING_LEVEL"] = "INFO"

For updates or issues, you can track or contribute to relevant GitHub issues: - Issue 13035 - Issue 6519 Feel free to refer to these documents to fine-tune your use case! If you need further assistance, please let me know.

in what context do you want to set the env vars? On a deployment, or during a flow run?

I am working with the

run_dbt_model

command(s) and I already have a

profiles.yml

, so using the

DbtCliProfile

Block (with my Snowflake credentials) is not ideal, as my profile already parses all relevant Snowflake credentials from environment variables. I was led to the solution that I should load a SnowflakeCredentials block and export the secrets as environment variables, so

dbt

could pick them up without any other changes. Could I simply load the block and write to

os.environ["SNOWFLAKE_USER"] = "1"

?

yeah, you might have to call

.get_secret_value()

on them depending on which property of the block you want to write out though

@Kevin Grismore, just to follow up re: dbt, are there any plans for a new dbt guide to be released covering the recent changes and additions?

Good question! The author of those changes is out on parental leave for a couple of weeks but if you have any questions on how to use it I'm sure I can take a look at the codebase and share my opinions.

Ok thanks! I'm just trying to get a handle on this file in the recent 2.20.2 release:

src/integrations/prefect-dbt/prefect_dbt/cli/commands.py

and here are the docs: https://docs.prefect.io/latest/integrations/prefect-dbt/cli/commands/ I believe

DbtCoreOperation

is no longer the primary reccomendation so I'm working with the various

run_dbt_x

methods