https://prefect.io logo
Join Slack
Powered by
Hi! Does anyone happen to know a good walkthrough/...
# prefect-dbt
h
Hi! Does anyone happen to know a good walkthrough/demo of setting up the prefect dbt core blocks so I can run a small dbt project from prefect Cloud? I took a look at all of the different dbt blocks, some of the documentation, and couldn’t quite piece it together.
s
Hey! What kind of issues are you running into? Most of the dbt core blocks are really pieces to create the 
profiles.yml
file. If you can look at the file the blocks are creating it might point you to the parts that aren't working properly.
Alternatively, if you already have a functioning 
profiles.yml
you can supply it, and skip blocks altogether with something like this:
Copy code
from prefect_dbt.cli.commands import DbtCoreOperation
from prefect import flow

@flow
def trigger_dbt_flow() -> str:
    result = DbtCoreOperation(
        commands=["dbt build -t prod"],
        project_dir="prefect_demo",
        profiles_dir="~/.dbt"
    ).run()
    return result

if __name__ == "__main__":
    trigger_dbt_flow()
h
I have a working profiles.yaml and a dbt project on my laptop, but I want to run it on k8s and so I need to store all my config somewhere that isn’t on the file system. So I looked at all the Prefect dbt blocks, there are 8 of them in total, some seem to be partially overlapping, I tried to add a dbt CLI profile block, was prompted to add a SnowflakeTargetConfig block, and it again asked to add a Snowflake Connector block. At that point it felt like a rabbit hole, and I couldn’t quite figure out the… totality.
s
That's good feedback. I should write something up on stringing all of the dbt blocks together. I think you're on the right track. The last thing you're missing is the Snowflake credentials block. Then all of those roll up into the CLI profile block so you only have to load one block when the time comes to run the flow
31 Views
Open in Slack
PreviousNext
Powered by