https://prefect.io logo
Title
i

itay livni

02/06/2020, 5:42 AM
Hi - I am getting lost in the documentation regarding deployment. What is the right path to deploy a flow to Fargate? Agent or Task? Thanks
c

Chris White

02/06/2020, 5:47 AM
Hi @itay livni - @josh knows the most about the AWS side of the house, but let me get you started. The Fargate Agent is definitely the simplest way to get started in Fargate and I recommend you start there with a standard
RemoteEnvironment
+ Docker storage for your Flow. In this case the Agent will create a single FargateTask for each flow run it submits. If you find that you want to configure the FargateTask for each flow run individually (e.g., very particular memory / resource requirements), then you should consider the
FargateTaskEnvironment
with the
FargateAgent
. This will result in two fargate tasks: one which will pull your Flow, and then the one that your
FargateTaskEnvironment
will create with your specifications
j

josh

02/06/2020, 2:37 PM
Yep that’s correct! A Flow w/ Docker storage registered to Prefect Cloud can then be spun up on AWS Fargate using the Fargate Agent https://docs.prefect.io/cloud/tutorial/docker.html#cloud-flow-with-docker https://docs.prefect.io/cloud/agent/fargate.html
i

itay livni

02/06/2020, 5:25 PM
@josh Without prefect the process to deploy an ecs task is (1) Compose a Docker file (2) Build image (3) Register Image (4) Deploy to fargate. With prefect its (1) register flow with cloud (2) And then run agent? The second question is: Are then two versions of the flow registered one local execution and one ecs? And lastly if I wanted no prefect cloud interaction, but wanted to use "prefects docker file" could I do that? Thank you in advance
j

josh

02/06/2020, 5:44 PM
The process for prefect in the same vein would be: 1. Write flow 2. Add Docker storage to your flow. You may optionally provide a custom Dockerfile if wanted 3. Register flow w/ Cloud. The Docker storage will be built here. (you may also optionally build ahead of time but is not needed) 4. Run fargate agent 5. Agent deploys flow to fargate There will be one version of that registration of the flow. The Flow that is stored in the Docker image in your registry and then there is metadata representing that flow in Cloud. I’m not sure what you mean by that last question
i

itay livni

02/06/2020, 6:08 PM
@josh Thank you. I know that was a lot. Let me take a second stab at it