Netanel Malka
03/26/2024, 8:44 AMprefect.yaml
:
pull:
- prefect.deployments.steps.set_working_directory:
directory: ~/company-data-engineering/prefect-data-engineering/src
deployments:
- name: iceberg-sanity
tags:
- maintenance
- de
entrypoint: flows/iceberg_sanity.py:iceberg_sanity
parameters:
p1: mountain
p2: true
p3:
- p4: 1
- p5: 2
schedule:
- cron: "0 0 * * *"
active: true
- name: data-purge
tags:
- maintenance
- de
parameters:
job_image: company-job-data-purge
job_resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 0.5
memory: 1Gi
job_envs:
RETENTION: 90 days
ALERTS_RETENTION: 90 days
entrypoint: flows/data_purge.py:data_purge
schedule:
- cron: "0 0 * * *"
active: true
I am running this command:
prefect --no-prompt deploy --all --pool default-kubernetes-pool --prefect-file prefect.yaml
But I get this output:
You have passed options to the deploy command, but you are creating or updating
multiple deployments. These options will be ignored.
Deploying all flows with an existing deployment configuration...
I have seen a reference in the docs that using deploy --all with multiple deployments :
https://docs.prefect.io/latest/guides/deployment/kubernetes/#define-a-prefect-deployment
Quoted from the docs:
To deploy your flows, ensure your Docker daemon is running first. Deploy all the flows with prefect deploy --all or deploy them individually by name: prefect deploy -n hello/default or prefect deploy -n hello/arthur.
It should be working? It that a bug?
Thanks.Serina
03/26/2024, 2:12 PMprefect --no-prompt deploy --all
because the work pool for each deployment is already set for each deployment in your prefect.yaml
For re-deploying to a different work pool, I think you could probably use templating.Netanel Malka
03/26/2024, 2:17 PMprefect --no-prompt deploy --all
without work-pool should work for more than one deployment?Netanel Malka
03/26/2024, 2:19 PMSerina
03/26/2024, 2:22 PM--all
means to deploy all existing deployments. We have a lot of ways you can specify which deployments to deploy. For example, to deploy deployments that match a pattern you can run:
prefect deploy -n my-flow/* -n *dev/my-deployment -n dep*prod
(more examples in those docs above)