[THEORETICAL QUESTION]
Hi everyone, new to DBT / Prefect and Data Engineering. I am currently building my first DE capstone project where I am pulling data from an API, and pushing it to BQ. Orchestration happens with Prefect. I want to use DBT to partition and prepare the data in BQ for Looker. I am having issues wrapping my head around how I can make my code reproducible for others on GitHub as I am using DBT cloud. I found repo from Anna but it does not fully answer my question on how to make the code reproducible (especially in terms of DBT credentials). What would be the best practices here? Would I need to explain them how to setup the DBT cloud as well or is there a better way?
✅ 1
s
Sean Williams
03/15/2023, 5:03 PM
Have you seen prefect-dbt? You can save your credentials to a block, so you don't have to have your credentials in your code on github.
v
Vincenzo
03/19/2023, 2:19 PM
@Sean Williams thanks for this, yes I have seen it. I assume that means that I would include the dbt-transformation step as a step within my flow / new deployment. I will have a look into this thanks.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.