Robert Phamle
06/27/2024, 8:42 PMjob_variables
in flow.deploy
Marvin
06/27/2024, 8:42 PMjob_variables
in flow.deploy
, you can use the job_variables
argument of the Python flow.deploy
method. Here is an example to illustrate this:
python
from prefect import flow
import os
@flow
def my_flow():
# Your flow logic here
pass
if __name__ == "__main__":
my_flow.deploy(
name="my-vertex-ai-deployment",
work_pool_name="vertex-ai", # This should be your Vertex AI work pool name
job_variables={
"env": {
"EXECUTION_ENVIRONMENT": os.environ.get("EXECUTION_ENVIRONMENT", "local"),
"MY_NOT_SO_SECRET_CONFIG": os.environ.get("MY_NOT_SO_SECRET_CONFIG")
},
"stream_output": False # Example to turn off streaming output
}
)
### References
1. Overriding Job Variables
2. Work Pools Concept
3. Vertex AI Integration
This script deploys a flow called my_flow
to a Vertex AI work pool, with specific job variables set in the deployment. Adjust the variables and flow logic as necessary to fit your use case.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by