Hello. Any plans in the future to have a terraform provider to manage organization resources? Things like workspaces, api keys, and things like that ? It would be pretty useful for enterprise that needs to manage their infrastructure with a fully automated IaC.

a little bit of a chicken and egg problem because you need to have a workspace URL and API key to interact with Prefect programmatically, and creating a workspace is a subject to quotas based on your plan e.g., you may not be able to create 5 new workspaces through terraform if you purchased only 3 but a totally understandable problem (it would be cool if possible), I believe a good middle ground is to be able to create all agents & blocks programmatically within a workspace and this is something I was trying to do in those 2 repos: • https://github.com/anna-geller/dataflow-ops • https://github.com/anna-geller/prefect-dataplatform and for agents, we have several terraform recipes here: https://github.com/PrefectHQ/prefect-recipes

I would be tempted to say that even workspaces could be created via terraform. The terraform provider could simply throw an error if you are over the maximum number of workspaces for your account

the problem was added, how the solution will look like is up to our platform experts 🙌 we get you, we understand the problem

👍 Thanks

+1 to this. I agree setting up extra Workspaces can be useful but priority for me would be to be able to deploy Agents, Automations, Notifications, Blocks, etc.. via terraform

Also, the dream would be to actually be able to create "deploments" using terraform. It would make the life of teams using terraform so much easier. Instead of having to manage a CI/CD on their side that actually perform deployment create and deployment apply commands from the CLI.

Jean, creating deployments doesn't provision any infra though, it only sends 1 API call with deployment metadata to Prefect so not sure if that's a Terraform use case but anyway we have an internal ticket for that, we'll update in #CKNSX5WG3 if we add a feature to do that. Thanks for this request and adding details

I know, but managing lifecycles of deployments (create, update, delete) is far from being easy using the cli/python. Having terraform for that would make it easy to manage deployments for flows. Meaning • Adding a terraform resource for a flow deployment would create a deployment in prefect • Modifying any attributes to the deployment would update the deployment • Removing that resource from terraform would delete the deployment Currently there's no "clean" way of managing the lifecycle of these deployments other than for example deleting all deployments during each release, then recreating them all for examples.

deployments is a tricky one. I agree with Jean that managing deployments with Python or the CLI only helps us deploying them and not managing their lifecycle. But in my case for example I struggle to see it working as terraform infrastructure, as we have data analysts deploying workflows all the time, and asking them to manage terraform sounds very out of their scope of work. We haven't migrated to 2.0 yet, but in 1.0 that's why we created a wrapper library on top of Prefect to automate some of these API calls and make it simpler for everyone

yeah, I have the same problem. Still I would prefer to have a state with the proper lifecycle and I could create a terraform module with really simple variables that data engineer can fill in that would create their flow automatically during our team deployment to production.

I'm a little late to the party here, but my main use case would be for creating blocks. Having a workspace with API keys set up ahead of time and provided as variables is fine. And I don't mind managing deployments separately either. But creating blocks like S3 storage and ECSTask should definitely be tightly coupled with the Terraform that creates these S3 buckets and ECS task definitions.