Hi, I would like to fish for a conceptual clarification and best practices around CI/CD in ML. There seems to me that there is a functional overlap between GitLab CI/CD and Prefect; and I have to conceptualize some sort of Continuous Integration and Continuous Delivery for machine learning, which I could put into Prefect dataflows.
• As I understand it, data is better passed between Prefect tasks.
◦ This would make Prefect a better candidate for running data and model validation tests.
• GitLab CI/CD is designed to test code.
◦ I am not sure if I should use it to run data and model validation tests.
◦ I think it has its place in integrating and delivering Prefect code.
I am slightly confused whether
1. GitLab CI/CD would end up testing the same things as Prefect would at some point
2. I can do without GitLab CI/CD
It is not clear how to use one or the other specifically.