New to using Prefect...there's gotta be something ...
# ask-communityk
Kevin Ruprecht
05/01/2023, 8:45 PMNew to using Prefect...there's gotta be something I'm missing in setting up. Deployed in GCP, using Prefect CLI and dbt and BigQuery. Just trying to write a simple flow for testing. I used the server UI to save a block for the link to my test dbt repo. The repo is public. I'm trying to reference that block in my flow, and I just keep getting an error that the project_dir path is invalid.
Kevin Ruprecht
05/01/2023, 8:45 PMs
Sean Williams
05/01/2023, 9:31 PMWhat happens if you change line 11 to
project_dir=github_block.get_directory(),k
Kevin Ruprecht
05/01/2023, 9:34 PMKevin Ruprecht
05/01/2023, 9:35 PMif i used
"github_block"Kevin Ruprecht
05/01/2023, 9:38 PMand the error i get using "github_block" for the project_dir is `ValueError: Since overwrite_profiles is True or profiles_path is empty, need
dbt_cli_profileKevin Ruprecht
05/01/2023, 9:38 PMwhich i don't really understand because i'm...using dbt_cli_profile??
s
Sean Williams
05/01/2023, 9:48 PMThat is a confusing message. I think it wants a new line below line 14, explicitly specifying
overwrite_profiles = Truek
Kevin Ruprecht
05/01/2023, 9:50 PMI get
profiles_dir value is not a valid pathKevin Ruprecht
05/01/2023, 9:51 PMi may need to try defining credentials/profile values manually instead of using blocks? just to make sure my definitions work
s
Sean Williams
05/01/2023, 9:52 PMAh, try changing
profiles_dirdbt_cli_profileSean Williams
05/01/2023, 9:53 PMdbt_cli_profile=dbt_cli_profilek
Kevin Ruprecht
05/01/2023, 9:53 PMOh ok. I will try that later tonight. Thank you.
👍 1
Kevin Ruprecht
05/02/2023, 2:39 AMTried it, and now I'm getting the same error as this thread
https://prefect-community.slack.com/archives/C0470L5UV2M/p1681118539881009. But I'm using BigQuery/GCP, not Snowflake.
Kevin Ruprecht
05/02/2023, 3:26 AMultimately, i think the problem is that my code for the dbtCoreOperation is not correctly utilizing the GitHub block. I may have gotten beyond the dbt profile issue, but trying to use a remote GitHub repo for the actual dbt code to execute doesn't seem to be working. Don't know why. I get this error
project_dir value is not a valid path (type=type_error.path)Kevin Ruprecht
05/02/2023, 3:47 AMI kinda think having prefect-related code and my dbt project code in the same repo would be advisable here.
145 Views