[DBT] [Prefect] Thanks for the relentless support ...
# data-tricks-and-tips
v
[DBT] [Prefect] Thanks for the relentless support everyone. I am amazed by Prefect and really want to dive deeper into it! I am trying to orchestrate my dbt job with prefect. Following these instructions I was able to set up the credentials block. Unfortunately the job is not executed and it throws no error ๐Ÿงต - somebody with a similar setup that encountered this issue as well?
Here my code
Copy code
from prefect import flow, task
from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import run_dbt_cloud_job
from prefect_dbt.cloud import DbtCloudCredentials, DbtCloudJob

your_dbt_api_key = "API"  # (1) insert your dbt Cloud API key
your_dbt_account_id = "ACCOUNT"  # (2) insert your dbt Cloud account ID
your_dbt_job_id = "JOB"  # (3) insert your dbt Cloud job ID
dbt_cloud_credential_name = "magic-the-gathering-credentials"

DbtCloudCredentials(api_key=your_dbt_api_key, account_id=your_dbt_account_id).save(
    dbt_cloud_credential_name, overwrite=True
)
dbt_cloud_credentials = DbtCloudCredentials.load(dbt_cloud_credential_name)


@task(log_prints=True, name="Run dbt Cloud Job")
def run_dbt_job_flow():
    """Runs the dbt Cloud job and returns the result"""
    result = run_dbt_cloud_job(
        dbt_cloud_job=DbtCloudJob.load(dbt_cloud_credentials, your_dbt_job_id),
        targeted_retries=5,
    )
    return result


@flow(log_prints=True, name="[Magic: The Gathering] Load latest data to BigQuery")
def dbt_job_orchestration():
    """Orchestrates the flow of the dbt Cloud job"""
    run_dbt_job_flow()
c
You may want to cross post to #C0470L5UV2M - smaller channel but specific to dbt
๐Ÿ™Œ 1
v
Thanks, I read that in the free tier, the DBT API is not accessible.. That changes a lot.
a
have you tried the
trigger_dbt_cloud_job_run_and_wait_for_completion()
here is an example that worked for me on the free tier
Copy code
from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import trigger_dbt_cloud_job_run_and_wait_for_completion        

trigger_dbt_cloud_job_run_and_wait_for_completion(
        dbt_cloud_credentials = DbtCloudCredentials.load("test-creds"),
        job_id = x,
        poll_frequency_seconds = 60
        )
๐Ÿ™Œ 1
need to call it at the end but that function should work
v
Thanks @Adam - for me this code results in the same: No reaction, No error logs.
a
do you have the cred block for dbt in the UI?
v
Yes, I create it via code, on top.
Final answer: The API in DBT is only usable if the account is paid for.
hence the tutorial is not for free tier users of dbt.
a
๐Ÿ˜ญ