Shaariq
02/21/2024, 5:13 AMjob_variables.env
variable at the moment however it's proving to be not particularly easy to dynamically have env variables per deployment. Note that I want to use the prefect.yaml
as little as possible as it does not fit with my current ci/cd processMarvin
02/21/2024, 5:14 AMprefect.yaml
configurations as much as possible, you can leverage environment variables in your deployment steps. Unfortunately, the documentation doesn't seem to cover the specifics on directly using Kubernetes secrets in-depth, but you can generally manage environment variables which can reference Kubernetes secrets within your deployment environment.
For example, you can use environment variables when running prefect deploy
which can be set in your CI/CD pipeline. Those environment variables can pull values from Kubernetes secrets, ensuring that sensitive information is not hardcoded in your deployment scripts.
Here is what the Prefect documentation mentions about using environment variables in deployment steps:
Prefect now supports the usage of environment variables in deployment steps, allowing you to access environment variables during theFor the implementation details, you can refer to the following pull request: prefecthq/prefect#10189. If your CI/CD environment allows you to dynamically set environment variables, you can map those to the Kubernetes secrets you need. In your deployment scripts, reference these environment variables which should contain the secret values pulled from your Kubernetes secret store. Using secrets directly in Kubernetes deployments typically involves referencing the secret in your deployment configuration (not specifically a Prefect feature) and then having your application or workflow configured to read from those secret-provided environment variables. I would recommend exploring the Kubernetes documentation on Secrets or discussing with your CI/CD pipeline provider for best practices on referencing Kubernetes secrets as environment variables within your deployment process. For detailed guidance and examples, you may consult the full Prefect guides on Kubernetes deployment. If there's more specific functionality related to Kubernetes secrets and Prefect that isn't covered in the documentation, you might consider reaching out to the Prefect community or opening a GitHub issue in the prefecthq/prefect repository for more targeted assistance.action at runtime or during thepull
andbuild
actions when runningpush
.prefect deploy
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by