Yaron Levi

07/21/2023, 8:37 AM
❄️ Snowflake Container Service ❄️ So I've watched some content from the Snowflake Summit and I saw that Astronomer (Airflow) will add support for using the new Snowflake Container Service as a the compute infrastructure for flows to run on. I'm wondering if Prefect is also planning to add such feature. It could be just another Infrastructure type like ECS Tasks and the others. I know for our team, and other colleagues, that this will be huge benefit and an attraction point towards Prefect. Many teams already use Snowflake, but don't have knowledge and time to setup an ECS Cluster and acquire additional knowledge of ECS Tasks and Services. We use ECS Tasks together with Prefect Cloud, and although ECS is considered a "high level" service with not much operations needed, it's still something you need to know, observe and understand. Using Snowflake as an infrastructure backend could hugely simplify the "infrastructure story" for Prefect (billing, security, compute closer to data...).