https://prefect.io logo
Title
f

fozan talat

05/02/2023, 6:57 AM
Hi Everyone šŸ™‚, I recently built an End-to-End Data Engineering project for analyzing Chicago's Divvy Bikeshare dataset, I used the following techstack for building this project. ā€¢ Google Cloud Storage (GCS) - as Data Lake ā€¢ Google BigQuery - for Data Warehouse ā€¢ Terraform - as Infrastructure-as-Code (IaC) tool ā€¢ Prefect - for orchestration ā€¢ dbt - for transformation and data modeling ā€¢ Google Looker studio - for visualizations please do check it out and don't forget to star the GitHub repo, and let me know what you think. link to repository : https://github.com/Fozan-Talat/divvy-bikeshare-de-project
šŸ’Ŗ 11
ā¤ļø 10
:upvote: 6
šŸ”„ 5
šŸš² 2
k

Kevin Ruprecht

05/02/2023, 2:58 PM
I like a lot of what I see in that repo, but I am curious. You commented out the dbt portion of your Prefect flow due to errors you were getting. What errors?
šŸ‘ 1
j

jawnsy

05/02/2023, 3:12 PM
This is really cool! Thanks for sharing
šŸ‘ 1
f

fozan talat

05/02/2023, 3:29 PM
@Kevin Ruprecht Finished in state Failed('Flow run encountered an exception. RuntimeError: PID 17072 failed with return code 1.\n'), I also posted in the channel prefect-dbt but got no response, so I completed it without triggering dbt from prefect.
k

Kevin Ruprecht

05/02/2023, 3:36 PM
Got it. thanks. I ask because I've been having problems trying to use Prefect blocks for the dbt project / profile paths. That error is not super-specific, but I think I got it at one point or another.