Hi everybody! We’d like to have the flow registration done automatically as part of our CI pipeline, and I’m curious to hear how others have approached this. I’m trying things out using a github action which runs on push an AWS ECS Task which runs a container in a Fargate cluster (our flows all run on Fargate using Docker storage) which then runs the registration method , but the aws setup is not as straight forward as I hoped. 😅 Pls let me know your solution approaches! 🙏
k
Kevin Kho
06/04/2021, 5:51 PM
Hey @Fina Silva-Santisteban, this is quite involved of a setup that we may not be able to help with. Maybe someone from the community can chime in. We also have a partner dedicated for infrastructure help if you want to talk to them
f
Fina Silva-Santisteban
06/04/2021, 6:04 PM
@Kevin Kho sure! Could you pls connect me with your partner? (Pls feel free to continue the convo over dm!) If anybody else has set up an automated registration process, it would be great to hear from you. simple smile
k
Kevin Kho
06/04/2021, 6:51 PM
@George Coyne
🙏 1
a
Austen Bouza
06/04/2021, 11:34 PM
@Fina Silva-Santisteban I just finished setting up flow registration and deployment with CI/CD for my company, also using ECS Fargate. I can’t share the source but I found this discussion to be very helpful when designing. We are generating a CloudFormation stack dynamically using Troposphere and using GitHub Actions to handle flow registration and storage, and to build and deploy flows as images to ECR/ECS.
🙏 1
👍 1
f
Fina Silva-Santisteban
06/07/2021, 4:09 PM
Thanks so much for your advice and for sharing that thread @Austen Bouza! That’s very helpful!!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.