Hello everyone, quick question about infra blocks vs workers, Is there an equivalent to `infra_overr...

Ouail Bendidi

05/15/2023, 7:20 AM

Hello everyone, quick question about infra blocks vs workers, Is there an equivalent to

infra_overrides

in workers ? We use a default config in an

ECSTask

and override with the docker image we want to use, Not sure how to have the same with workers ?

✅ 1

Ouail Bendidi

05/15/2023, 7:21 AM

Copy code

Deployment.build_from_flow(
        name="manual",
        flow=my_flow,
        skip_upload=True,
        apply=True,
        load_existing=False,  # always overwrite
        infrastructure=ECSTask.load("default"),
        infra_overrides={"image": "xxxx", "memory": "xxxx", "cpu": "xxxx"},
    )

Jamie Zieziula

05/15/2023, 2:26 PM

Are you using projects? You can define a

job_image

to override the default image set on your work pool under the work_pool configuration of your deployment:

Copy code

work_pool:
      name: kubernetes-work-pool
      job_variables: 
        image: your-image-here

Jamie Zieziula

05/15/2023, 2:27 PM

you can also edit your deployment in the ui and provide infra_overrides there

Ouail Bendidi

05/15/2023, 2:30 PM

cool thanks. We are not using projects (Pretty costly to migrate our existing workflows to use projects for now), can I set the

job_variables

from the python API directly ?

Jamie Zieziula

05/15/2023, 4:13 PM

you can set the image from the infra_overrides parameter in the Deployment.build_from_flow

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.