Hi Folks! I'm playing around with prefect core, a...
# prefect-community
Hi Folks! I'm playing around with prefect core, and I'd love some example of etl doing: 1. executing a select query in a database server and retrieving the results 2. transforming some fields in each row 3. writing to a CSV file I was wondering what's the proper way to build this and have some questions also: • Would this case scale to milions of rows if using prefect? • Should I orchestrate with prefect and use a library for etl like [bonobo](https://www.bonobo-project.org)? • Can I separate the e/t/l steps in tasks and reuse for writing to AWS S3 for example? Regards Juarez
Hi @Juarez Rudsatz! Apologies for the delayed response here; Prefect is entirely agnostic to what infrastructure you run on and what python logic is executed within your tasks, so your goals here align very nicely with what Prefect provides. Moreover, external libraries generally play nice with Prefect so if Bonobo provides a convenient API for your work, you should definitely use it alongside Prefect. For more about writing and re-using tasks, check out this tutorial: https://docs.prefect.io/core/tutorials/task-guide.html