Shaariq
02/12/2024, 5:57 AMflows
a_service
src
flow.py #flow will use utils.py
utils.py
requirements.txt # custom requirements
Dockerfile # standard dockerfile for building custom image
another_service
src
flow.py #flow will use utils.py
utils.py
requirements.txt # custom requirements
Dockerfile # standard dockerfile for building custom image
now, I want to be able to deploy these from outside the 'service context'.
Basically, I can have a separate function that creates a RunnderDeployment
and creates a deployment that way. However, there are a few problems:
• I lose the context of the entrypoint (although I could fix that by having a standard path to the flow
?
• More importantly, I lose the parameters and this is the main part that I am stuck on. Without me having to manually define the parameters, is there a way for me to fetch them for a given flow when it's already just built into an image?
here is an example of the deployment code:
flow_images = [
"testing/flow:latest",
]
for f in flow_images:
d = RunnerDeployment(name="testing", flow_name="testing")
deploy(
d,
image=f,
build=False,
push=False,
work_pool_name="default",
)
I realize this is probably niche! However I believe it to be a very scalable way to write flows. I am just missing some pieces of the puzzle and would love some insight 🙂