https://prefect.io logo
Title
k

Kevin Ruprecht

05/01/2023, 8:45 PM
New to using Prefect...there's gotta be something I'm missing in setting up. Deployed in GCP, using Prefect CLI and dbt and BigQuery. Just trying to write a simple flow for testing. I used the server UI to save a block for the link to my test dbt repo. The repo is public. I'm trying to reference that block in my flow, and I just keep getting an error that the project_dir path is invalid.
Screen Shot 2023-05-01 at 3.45.30 PM.png
s

Sean Williams

05/01/2023, 9:31 PM
What happens if you change line 11 to
project_dir=github_block.get_directory(),
?
k

Kevin Ruprecht

05/01/2023, 9:34 PM
Screen Shot 2023-05-01 at 4.33.28 PM.png,Screen Shot 2023-05-01 at 4.33.51 PM.png
if i used
"github_block"
as the value for project_dir, that at least moved the error to profiles_dir
and the error i get using "github_block" for the project_dir is `ValueError: Since overwrite_profiles is True or profiles_path is empty, need
dbt_cli_profile
to write a profile`
which i don't really understand because i'm...using dbt_cli_profile??
s

Sean Williams

05/01/2023, 9:48 PM
That is a confusing message. I think it wants a new line below line 14, explicitly specifying
overwrite_profiles = True
k

Kevin Ruprecht

05/01/2023, 9:50 PM
I get
profiles_dir value is not a valid path
i may need to try defining credentials/profile values manually instead of using blocks? just to make sure my definitions work
s

Sean Williams

05/01/2023, 9:52 PM
Ah, try changing
profiles_dir
to
dbt_cli_profile
dbt_cli_profile=dbt_cli_profile
k

Kevin Ruprecht

05/01/2023, 9:53 PM
Oh ok. I will try that later tonight. Thank you.
👍 1
Tried it, and now I'm getting the same error as this thread https://prefect-community.slack.com/archives/C0470L5UV2M/p1681118539881009. But I'm using BigQuery/GCP, not Snowflake.
ultimately, i think the problem is that my code for the dbtCoreOperation is not correctly utilizing the GitHub block. I may have gotten beyond the dbt profile issue, but trying to use a remote GitHub repo for the actual dbt code to execute doesn't seem to be working. Don't know why. I get this error
project_dir value is not a valid path (type=type_error.path)
I kinda think having prefect-related code and my dbt project code in the same repo would be advisable here.