https://prefect.io logo
p

Parwez Noori

09/21/2022, 12:48 PM
Hi, I am trying to solve a problem with different repos using prefect 2. I have a repo for prefect itself and then repos for the DS projects. Each have virtual environments setup. I have simple flows setup that: 1. Creates a yaml file for each deployment and applying a schedule for each deployment visible in prefect cloud. 2. Run the scheduled tasks as k8s jobs. However when I'm pushing deployments from the DS repo it is registered but never actually run/triggered. I have also created another storage block for DS project, but this has not solved the problem.
k

Khuyen Tran

09/21/2022, 3:16 PM
Did you start the agent with the same work queue as the deployment when running the deployment?
p

Parwez Noori

09/22/2022, 7:47 AM
Yes I did.
k

Khuyen Tran

09/22/2022, 3:34 PM
Have you tried to use the process infrastructure to see if the deployment runs?
p

Parwez Noori

09/27/2022, 7:26 AM
No I will do that now!
Hi @Khuyen Tran it does not run as an process either. What else can be done?
k

Khuyen Tran

10/04/2022, 2:28 PM
Can you paste the command you use to create the deployment?
p

Parwez Noori

10/05/2022, 1:55 PM
prefect deployment build run_model.py:run_xgboost
Copy code
-n run_model_xgboost
-q kubernetes
Copy code
-i kubernetes-job
-t model-run
Copy code
-sb azure/azure-block
-o yaml-files/run_xgboost.yaml
Copy code
--cron "15 8,9 * * *"
--override image=_.azurecr.io/__ ` --override namespace=prefect2
prefect deployment apply ./yaml-files/run_xgboost.yaml
k

Khuyen Tran

10/05/2022, 1:57 PM
And what is your agent run command like?
p

Parwez Noori

10/05/2022, 1:59 PM
command: - prefect - agent - start - '-q' - kubernetes
My normal deployments from the "prefect repo" works great.
However, as soon as I'm in another path on my local pc then it does not work.
C:\Users\username\github\prefect_repo works C:\Users\username\github\data_science does not
The deployment is visible in UI, but when it should actually run the deployment. It does never run.
k

Khuyen Tran

10/05/2022, 2:53 PM
It should work independent of the name of the path. Is there a difference in dependencies between prefect_repo and data_science?
1
p

Parwez Noori

10/05/2022, 3:06 PM
YES Khuyen!
Thank you a lot. There was a difference between the dependencies.
Appreciate the help!!! 🙌
3 Views