https://prefect.io logo
Join Slack
Powered by
Hello! With prefect-dbt are the only connection ty...
# prefect-dbt
q
Hello! With prefect-dbt are the only connection types allowed those that are in 
prefect_dbt.cli.configs
(snowflake bigquery postgres) or is there a more general option where someone could connect to something like clickhouse
s
In this case, it'll be easier to supply a profile path that points to an existing 
profiles.yml
file. I just skimmed your post in dbt's slack, and it sounds like you could solve your env var/secret problem by loading Prefect secrets and setting them as an environment variables before you run the dbt job.
What version of 
prefect-dbt
are you using?
q
Copy code
Name: prefect-dbt
Version: 0.4.3
I can use a profile path and pass the path to profiles.yml, but I'm not able to get the environment vars to show up when adding them to the work pool
s
Are you setting the environment variables in prefect.yaml? If so, do you see them in the UI when you look at the workpool?
If you're passing clickhouse credentials, you might want to use a Secret block. You could set them in your flow code before running the dbt task. For example:
Copy code
secret_block = Secret.load('test-secret')

os.environ['clickhouse-cred'] = secret_block.get()
👀 1
q
And yes that is there, but not pulling. I will try to set in flow code
Okay, setting environment variable seems to have worked, though now I am hitting a new error.
Copy code
20:08:31.643 | INFO    | Flow run 'adorable-boar' - PID 68 stream output:
20:08:31  Encountered an error:
Runtime Error
  Credentials in profile "analytics", target "dev" invalid: Runtime Error
    Could not find adapter type clickhouse!
s
Progress!
I think you need to pip install 
dbt-clickhouse
first
q
Haha, that might help
Thank you
Its working!
success kid 1
Thank you for your help
46 Views
Open in Slack
PreviousNext
Powered by