https://prefect.io logo
#prefect-aws
Title
# prefect-aws
l

Lawrence Lee

01/06/2023, 8:55 AM
I have been working through creating an ECSTask as an infrastructure block for flow deployments and through extensive trial and error found some observations that were not part of the documentation: 1. If setting a
task_definition
,
containerDefinitions.name
must be “prefect”. Otherwise the new registered task definition will contain two separate
containerDefinitions
items and runs will fail. 2. I believe that the
command
parameter has to be blank/null, otherwise your flow will not be run properly. 3. If using the
task_customizations
parameter, you must cast your array(dict) as a
JsonPatch
object otherwise the pydantic validation will fail. This conflicts with the documentation and the examples in the
dataflow-ops
repo (@Anna Geller)
Copy code
task_customizations=JsonPatch([
        {
            "op": "replace",
            "path": "/networkConfiguration/awsvpcConfiguration/assignPublicIp",
            "value": "DISABLED",
        },
    ])
a

Anna Geller

01/06/2023, 3:05 PM
#1 thanks so much, I'll add this #2 it will be overridden by infrastructure block anyway, so this shouldn't matter - if you see some behavior that doesn't match your expectation, please submit a GitHub issue with a minimal reproducible example and tag me, I'll reproduce and ask the team #3 I couldn't reproduce, works as it
z

Zanie

01/06/2023, 4:07 PM
#2 is not overridden. If you specify a command, we will use it instead of our default. You can use this to run commands then enter the Prefect engine yourself.
a

Anna Geller

01/06/2023, 4:35 PM
thx Michael and sorry that I'm spreading the wrong info here but @Zanie I had an image for the agent with an entrypoint:
Copy code
ENTRYPOINT ["prefect", "agent", "start", "-q", "default"]
and I was able to use the same image for my flow run container on infra block without specifying any command. I guess my misinterpretation was because: • regardless of what entrypoint is set in Dockerfile, if some command is explicitly set on the infra block it will be used • if no command is set, we take this base run command correct or still wrong?
essentially I mean that we always override the command with either infra default's base command or user's explicit command
z

Zanie

01/06/2023, 4:38 PM
well there’s a difference between the ENTRYPOINT on the image and the
command
set on the
DockerContainer
block
a

Anna Geller

01/06/2023, 4:39 PM
this line is what I meant: we never use the one set in Dockerfile, we always override:
Copy code
"command": self.command or self._base_flow_run_command()
z

Zanie

01/06/2023, 4:40 PM
Well generally Docker containers run things as ENTRYPOINT <COMMAND>
👍 1
4 Views