Newb question even though I have 4 deployments running and too much trauma from trying to get them t...
j
Newb question even though I have 4 deployments running and too much trauma from trying to get them to work 🙃 TL:DR below I'm trying to set up some dbt-core deployments on an EC2 process worker. I have a single repo in Github that has dbt projects nested in a folder. To even run
prefect deploy
I need to have my code on the worker instance which means I have to clone it. I clone it and run
prefect deploy --prefect-file ./dbt-core/prefect.yml
and walk through the steps. My issue is that I want it to pull the same repository the prefect.yml is in, but it instead always clones the repo again into another folder and pulls that. My prefect.yml is pointing to the entrypoint in the code that already existed because it has to already exist when you run
prefect deploy
. The problem there is that now it only runs the old code from the repo that is not being pulled since that's where the entrypoint is. TL;DR - what's the recommended way to deploy on an EC2 instance when I want the entrypoint script and the prefect.yml to be in the repo it pulls in the pull step? Should I have a separate script that does the deployment and I just FTP or copy paste it onto the instance and run it? That seems crazy to me but I'm terrible at setting up deployments and don't know the best way to do it.
For posterity: The solution we settled on was to move away from using the prefect.yml for the deployments and instead do it all in a separate Python script that just does the deployment and/or serving.