Hello! I am trying to implement a DBT pipeline, with a custom sequence of dbt commands, from a dbt p...
s
Hello! I am trying to implement a DBT pipeline, with a custom sequence of dbt commands, from a dbt project in Prefect. I have manage to have it running with both prefect_dbt and prefect_dbt_flow. However, the prefect_dbt library (which is able to run the custom commands) does not seem to generate the DAG in the flow run, while the prefect_dbt_flow does not allow me to specify custom commands (I think?). Have I missed something, or is it not possible to combine the two?
s
I don't think you'd want to try to combine the two.
prefect-dbt-flow
will display each node at the expense of overriding how dbt would normally execute the nodes.
prefect-dbt
won't give you that level of granularity visually, but won't interfere with dbt's execution. It also offers an artifact (example, here) for more details about your dbt runs.
👍 1
s
Thanks a lot, this helped clear some stuff up 🙏 The prefect-dbt method with run summary artifact seems to be the more viable approach given the limitations in dbt-flow.