Adam
03/27/2023, 4:33 PMNate
03/27/2023, 6:07 PMprefect agent start
, an agent would attempt to submit flow runs to docker desktop (pulling the image
specified on your infra block)
so for example, you could install docker on a vm, set PREFECT_API
env vars, and do prefect agent start
(prob via something like systemd) and you could start running containerized flow runs on that VM
just a side note that if you're a k8s user, the helm chart is the easiest way to get going very quicklyAdam
03/27/2023, 6:14 PMJarvis Stubblefield
03/27/2023, 6:19 PMNate
03/27/2023, 6:21 PMappreciate the help again! been asking a lot latelyanytime!
ECSTask
is a different type of infra block which would replace DockerContainer
, although they are similarly rooted in containerization π (as in, you could docker build / push
your image to ECR in a way that doesnt involve Prefect's DockerContainer
at all, and then use ECSTask
infra block)
helm is for those who bring their own k8s cluster (managed or otherwise), which doesn't sound like your situation if you're looking at ECS
regardless of eventual runtime infra, you'll need an agent sitting somewhere constantly asking "do I have any work to submit?". in the case of ECS, that can pretty much be anywhere if you attach your AWSCredentials
to the ECSTask
blockAdam
03/27/2023, 6:59 PMNate
03/27/2023, 7:34 PMcustom build the other if thats what we go withhmm I'm not certain what you mean but just to reiterate in case thats helpful, basically each deployment (at this time) needs a single infra block, which can be any of the valid infra block types (including DockerContainer and ECSTask) - multiple of which are docker-based in nature. in any docker-based runtime, you'll need to build a custom image if you have custom dependencies outside of the prefect base image - for ECS, it makes sense to push your custom image to ECR, but its totally up to you / your objectives
Adam
03/27/2023, 8:52 PMNate
03/27/2023, 8:54 PMAdam
03/27/2023, 9:23 PMRitabrata Moitra
05/11/2023, 6:16 AM