We're starting work on building out our Prefect instance, and have been debating internally on how to manage to organize/structure/run our jobs that span multiple repos & images. Right now have the following repos:
⢠Prefect (just some simple flows so far)
⢠DBT
⢠App
What we could use some advice on is how to tie all of these together. Both DBT and App have Docker images they run on. Our simple user case is to run a DBT flow and then [once done] run an App flow. For those two Repos, I think the right move is deploy both of those flows/images separately, and then call each separately like this?
I think this is the lowest friction way to do it, yeah. It's also possible to have just the dbt deployment have a schedule, and make an automation (prefect cloud only) that runs your app deployment once it completes
šš» 1
e
Eric Albanese
03/20/2024, 8:51 PM
Cool, makes sense! Thanks Kevin
m
Miguel Silva
03/20/2024, 10:14 PM
Hello Eric. How did you run your dbt flow? Does it run from the dbt_project.yml?
I'm having trouble deploying dbt locally
Thanks Eric. I tried following that guy, sounds simple enough, right? But i get stuck with a runtime error
" RuntimeError: PID 22088 failed with return code 1. "
Miguel Silva
03/22/2024, 11:12 AM
Nevermind, I swap to prefect-dbt-flow and its working now!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.