Adriano Resende
12/27/2024, 12:54 PMMarvin
12/27/2024, 12:55 PMName: database_host
Value: localhost
Name: database_port
Value: 5432
Name: log_level
Value: INFO
Important notes (because there are always complications):
- These variables are visible to anyone with access to your workspace
- They are NOT encrypted, so please don't store sensitive data here
- For sensitive information, use Secret blocks instead (I cannot stress this enough)
You can find all the dreary details in the Variables documentation.
Just between us, if you're dealing with sensitive data, you might want to look into the Secrets functionality instead. Though that's a story for another question, isn't it?
Would you like to know how to actually use these variables in your flows, or shall we leave it at that?Adriano Resende
12/27/2024, 12:59 PMAdriano Resende
12/27/2024, 1:00 PMMarvin
12/27/2024, 1:00 PMenv:
DATABASE_URL: <postgresql://localhost:5432/mydb>
API_KEY: ${secrets.MY_API_KEY}
LOG_LEVEL: INFO
A few depressing but necessary notes:
- The format is YAML-based
- You can reference secrets using the ${secrets.SECRET_NAME}
syntax (which is actually quite useful for keeping sensitive data out of plain sight)
- These environment variables will be available to all flow runs in this work pool
The exact location of the environment variables section might vary slightly depending on your work pool type, but it's typically in the job configuration or base job template section.
In my vast and perpetually gloomy experience, I should mention that if you're using sensitive information, you really should be using Prefect's secrets management instead of putting raw values in environment variables. But I suppose that's a story for another existentially challenging day.
Would you like to know how to override these environment variables at the deployment level, or shall we end this particular journey into infrastructure configuration here?
Reference: Work Pools documentationBring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by