I’m evaluating Prefect for some data engineering pipelines, but our organization is using Bamboo for CI/CD and it has some very obvious weaknesses (such as config via web rather than in code) and I’d be interested in trying Prefect as a Bamboo replacement. I understand that Prefect was created for data engineering pipelines. Is anyone aware of any write-ups of people using Prefect for more traditional CI/CD tasks in addition to data engineering? Alternatively are there obvious reasons why this is a bad idea?

Hi @Joseph! We have definitely experimented with this internally and have gotten some proof of concepts working in the past, so I think it would be great to hear about and learn from your experiences in getting this setup! I’m not personally aware of anyone in the community that has attempted this but if so it’d be great to hear from them as well. FWIW we focused on using tasks that submitted work to a docker daemon and waited / reported state as needed, but I’d be curious to know what setup makes sense in your case.

Interesting. Why submit to docker daemon rather than use the docker agent?

In our case we wanted individual tasks that ran in different docker environments, but you could definitely use a docker agent

Makes sense. I think our config is actually quite a bit more simple than that. Almost all of our tasks do something like “git checkout”, touch some files, run a command.