Hi All, I have this ELT scenario and appreciate that you could give me suggestions. 1: ET runs as first step that pulls data from a SQL server, saves data into zipped csv files and then uploads them into snowflake by PowerShell scripts. 2: After EL is done, T runs through a dbt project 3: Then we need to process tabular cubes and Power BI datasets through PowerShell scripts and API. I'm thinking a dedicated image for each step and then use prefect to create containers in a workflow, run each step as a task insider this container one by one. and then shutdown containers in the flow's last step. Is that possible? Another way is that creating flow for each step and another "parent" flow to mange the workflow.

I think that a single image for this is fine, as long as you don’t have any dependency conflicts between each of those flows. Not to blatantly promote my own blog posts, but there is this 3-part blog post series about exactly this use case: • https://www.prefect.io/blog/flow-of-flows-orchestrating-elt-with-prefect-and-dbt/ • https://www.prefect.io/blog/orchestrating-elt-on-kubernetes-with-prefect-dbt-and-snowflake-part-2/ • https://www.prefect.io/blog/orchestrating-elt-with-prefect-dbt-cloud-and-snowflake-part-3