Ethan Homan11/02/2022, 7:09 PM
It would be great to provision a small Fargate task with a Docker image with just the DBT project for the first one, and then provision an ECS task with significantly larger resources and a different Docker image for the ML training job. Is there a way to do this by creating dependencies between Prefect deployments or by specifying infrastructure at the task or sub-flow level?
(DBT job on Snowflake) -> (ML training job).
Kevin Grismore11/02/2022, 7:13 PM
). You can deploy flows that need their own infrastructure, like docker images, and then run those deployments from another flow.
from prefect.deployments import run_deployment
Ethan Homan11/02/2022, 7:14 PM