Sean Talia01/28/2021, 5:30 PM
) to the flow considered to be canonical? i'm trying to weigh the pros and cons of having some other process like terraform handle management of the task definition (thereby only leaving the ARN to be supplied to prefect) vs. using prefect to handle that
being out of sync with what your prefect flow has in its definition?
Jim Crist-Harif01/28/2021, 6:02 PM
if possible (it's the simplest for you to manage).
is good if you don't want to store the task definition in prefect cloud's database,
is good if you want to control when and how the task definition is registered (and lets you avoid giving the agent permission to register tasks). If you don't have super strict security requirements though,
is probably the most user friendly.
Sean Talia01/28/2021, 6:50 PM
makes it sound like:
ECS Tasks are composed of task definitions and runtime parameters.
flow.run_config = ECSRun( task_definition="<s3://bucket/path/to/task.yaml>", image="example/my-custom-image:latest", cpu="2 vcpu", )
Jim Crist-Harif01/28/2021, 7:01 PM
(and all run-configs) are for specifying where your flow process runs. Your flow code itself is stored in your specified
, and contains the DAG of your flow.
Sean Talia01/28/2021, 7:08 PM
Jim Crist-Harif01/28/2021, 7:18 PM
run config (and all run configs) are for specifying how to setup and deploy a process for running a prefect flow. The task definition used by an
must be used for running a flow run, not some arbitrary task. If you're trying to use prefect to manage arbitrary ECS tasks, you'll need to write the logic for starting (or registering or monitoring or ...) as prefect tasks and encode them into a larger flow. For example, you might write a prefect flow that runs in a single ECS task (described by
) that kicks off and monitors other arbitrary ECS tasks that you wrote yourself.
Sean Talia01/28/2021, 7:28 PM