Testing / CICD with Prefect and IAC
----
Hi guys 👋 👋 👋,
Hope you're all well! I'm currently working on a prefect cloud deployment using pulumi to declare the underlying workpool in AWS ECS. Looking for some guidance on testing as we want to test all changes in CI / pull requests to main. We can spin up and down the pulumi infra on a pull request due to it's idempotency and thus have multiple branches open at once. We were hoping to do the same with the deployment of the prefect orchestrator itself as well. For this the thought was to create a new workspace for testing. If so, is it possible to create a workspace via api request, we will also be using the pro account?
If this isn't possible we were planning on deploying prefect locally during CI via a docker-compose setup and deploying the flows etc. into that for testing. Any resources / documentation on this would also be appreciated if this is a suggested method.
Cheers,
Mark
k
Kevin Grismore
07/23/2024, 4:36 PM
There is a create workspace route but I don't see one for deleting a workspace, probably because of the implications of what other data would be deleted by programmatically deleting a workspace.
Kevin Grismore
07/23/2024, 4:36 PM
What about creating and deleting a work pool in an existing workspace meant for testing instead?
m
Mark Cottam
07/28/2024, 5:56 PM
Hi Kevin, thanks for your response! Yes that could be a solution, setting up afew workspaces for testing so can achieve parallelism.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.