Chris Hansen
07/28/2022, 10:52 PMTaylor Curran
07/28/2022, 10:59 PMProcess
runs flows in a local subprocess.
• DockerContainer
runs flows in a Docker container.
• KubernetesJob
runs flows in a Kubernetes Job.Chris Hansen
07/28/2022, 11:10 PMTaylor Curran
07/28/2022, 11:26 PMChris Hansen
07/28/2022, 11:47 PMTom Klein
07/28/2022, 11:54 PMprefect agent run
?
in prefect 1.0, prefect itself provided an easy way to do it:
prefect agent kubernetes install -k API_KEY | kubectl apply --namespace=my-namespace -f -
(this is taken directly from the prefect 1.0 docs)
Have things regressed so that now users need to manually create & deploy a container in k8s that would start the agent, ourselves?Christopher Boyd
07/28/2022, 11:58 PMIlya Galperin
07/29/2022, 12:15 AMTom Klein
07/29/2022, 12:15 AMyaml
files rather than to rely on python config-as-code? for example, why shouldn't the different deployment(s) of each flow just be first-class python objects that are registered the same way flows were registered in 1.0, e.g. via a simple python API? is that something that's planned and just not ready yet? or not planned at all?
• can you provide an example for this manifest?Anna Geller
07/29/2022, 12:27 AMChristopher Boyd
07/29/2022, 12:44 AMapiVersion: apps/v1
kind: Deployment
metadata:
name: cloud2-agent
namespace: orion
spec:
selector:
matchLabels:
app: prefect-agent
replicas: 1
template:
metadata:
labels:
app: prefect-agent
spec:
containers:
- name: agent
image: prefecthq/prefect:2.0.0-python3.9
command: ["prefect", "agent", "start", "kubernetes"]
imagePullPolicy: "IfNotPresent"
env:
- name: PREFECT_API_URL
value: <https://api.prefect.cloud/api/accounts/><account id>/workspaces/<workspace id>
- name: PREFECT_API_KEY
valueFrom:
secretKeyRef:
name: prefectapi
key: PREFECT_API_KEY
---
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: Role
metadata:
namespace: orion
name: flow-runner
rules:
- apiGroups: [""]
resources: ["pods", "pods/log", "pods/status"]
verbs: ["get", "watch", "list"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: [ "get", "list", "watch", "create", "update", "patch", "delete" ]
---
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: RoleBinding
metadata:
name: flow-runner-role-binding
namespace: orion
subjects:
- kind: ServiceAccount
name: default
namespace: orion
roleRef:
kind: Role
name: flow-runner
apiGroup: <http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>
Tom Klein
07/29/2022, 12:47 AMChristopher Boyd
07/29/2022, 12:48 AMTom Klein
07/29/2022, 12:50 AM