Robert Phamle
06/27/2024, 8:42 PMjob_variables in flow.deployMarvin
06/27/2024, 8:42 PMjob_variables in flow.deploy, you can use the job_variables argument of the Python flow.deploy method. Here is an example to illustrate this:
python
from prefect import flow
import os
@flow
def my_flow():
# Your flow logic here
pass
if __name__ == "__main__":
my_flow.deploy(
name="my-vertex-ai-deployment",
work_pool_name="vertex-ai", # This should be your Vertex AI work pool name
job_variables={
"env": {
"EXECUTION_ENVIRONMENT": os.environ.get("EXECUTION_ENVIRONMENT", "local"),
"MY_NOT_SO_SECRET_CONFIG": os.environ.get("MY_NOT_SO_SECRET_CONFIG")
},
"stream_output": False # Example to turn off streaming output
}
)
### References
1. Overriding Job Variables
2. Work Pools Concept
3. Vertex AI Integration
This script deploys a flow called my_flow to a Vertex AI work pool, with specific job variables set in the deployment. Adjust the variables and flow logic as necessary to fit your use case.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by