Ilya Galperin
09/09/2022, 6:11 PMJean Luciano
09/09/2022, 6:45 PMAgent started! Looking for work from queue(s): workque-chosen...
Ilya Galperin
09/09/2022, 6:47 PMJean Luciano
09/09/2022, 7:17 PMIlya Galperin
09/09/2022, 7:33 PMChristopher Boyd
09/12/2022, 5:13 PMIlya Galperin
09/12/2022, 5:26 PMprefect-agent-8b76465df-sq4g7 agent Starting v2.3.2 agent connected to
prefect-agent-8b76465df-sq4g7 agent <https://api.prefect.cloud/api/accounts/ACCOUNTID/work>
prefect-agent-8b76465df-sq4g7 agent spaces/WORKSPACEID...
prefect-agent-8b76465df-sq4g7 agent
prefect-agent-8b76465df-sq4g7 agent ___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
prefect-agent-8b76465df-sq4g7 agent | _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
prefect-agent-8b76465df-sq4g7 agent | _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
prefect-agent-8b76465df-sq4g7 agent |_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
prefect-agent-8b76465df-sq4g7 agent
prefect-agent-8b76465df-sq4g7 agent
prefect-agent-8b76465df-sq4g7 agent Agent started! Looking for work from queue(s): main...
However, no flows from our “main” queue are being picked up by this agent which makes me think there might be a communication problem between the agent and the server, whether it’s an incorrect API key or something else. Is there some other logs or mechanism we can use to validate that it is connecting successfully… you mentioned API logs?Christopher Boyd
09/12/2022, 6:07 PMIlya Galperin
09/12/2022, 6:09 PMChristopher Boyd
09/12/2022, 6:09 PMIlya Galperin
09/12/2022, 6:10 PMChristopher Boyd
09/12/2022, 6:10 PMIlya Galperin
09/12/2022, 6:11 PMapiVersion: apps/v1
kind: Deployment
metadata:
name: prefect-agent
namespace: prefect2
spec:
selector:
matchLabels:
app: prefect-agent
replicas: 1
template:
metadata:
labels:
app: prefect-agent
spec:
serviceAccountName: default
containers:
- name: agent
image: prefecthq/prefect:${var.prefect_tag}
command: ["prefect", "agent", "start", "-q", "main"]
imagePullPolicy: "IfNotPresent"
env:
- name: PREFECT_API_URL
value: ${var.prefect_api_url}
- name: PREFECT_API_KEY
valueFrom:
secretKeyRef:
name: prefect-cloud
key: api_key
${var.prefect_tag}
in this case resolves to “2.3.2-python3.10”Christopher Boyd
09/12/2022, 6:13 PMIlya Galperin
09/12/2022, 6:13 PMChristopher Boyd
09/12/2022, 6:24 PMapiVersion: apps/v1
kind: Deployment
metadata:
name: orion
namespace: prefect2
spec:
selector:
matchLabels:
app: orion
replicas: 1 # We're using SQLite, so we should only run 1 pod
template:
metadata:
labels:
app: orion
spec:
containers:
- name: api
image: prefecthq/prefect:2.3.2-python3.10
command: ["prefect", "orion", "start", "--host", "0.0.0.0", "--log-level", "WARNING"]
imagePullPolicy: "IfNotPresent"
ports:
- containerPort: 4200
- name: agent
image: prefecthq/prefect:2.3.2-python3.10
command: ["prefect", "agent", "start", "kubernetes"]
imagePullPolicy: "IfNotPresent"
env:
- name: PREFECT_API_URL
value: <https://api.prefect.cloud/api/accounts/><accountid>/workspaces/<workspaceid>
- name: PREFECT_API_KEY
value: bad_value
---
apiVersion: v1
kind: Service
metadata:
name: orion
namespace: prefect2
labels:
app: orion
spec:
ports:
- port: 4200
protocol: TCP
selector:
app: orion
---
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: Role
metadata:
namespace: prefect2
name: flow-runner
rules:
- apiGroups: [""]
resources: ["pods", "pods/log", "pods/status"]
verbs: ["get", "watch", "list"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: [ "get", "list", "watch", "create", "update", "patch", "delete" ]
---
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: RoleBinding
metadata:
name: flow-runner-role-binding
namespace: prefect2
subjects:
- kind: ServiceAccount
name: default
namespace: prefect2
roleRef:
kind: Role
name: flow-runner
apiGroup: <http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>
prefect.exceptions.PrefectHTTPStatusError: Client error '401 Unauthorized' for url '<https://api.prefect.cloud/api/accounts/><redact>/workspaces/<redact>/work_queues/name/kubernetes'
Response: {'detail': 'Invalid authentication credentials'}
For more information check: <https://httpstatuses.com/401>
An exception occurred.
Configure Prefect to communicate with the server with:
prefect config set PREFECT_API_URL=<http://0.0.0.0:4200/api>
View the API reference documentation at <http://0.0.0.0:4200/docs>
Check out the dashboard at <http://0.0.0.0:4200>
INFO: Started server process [9]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on <http://0.0.0.0:4200> (Press CTRL+C to quit)
Agents now support multiple work queues. Instead of passing a single argument,
provide work queue names with the `-q` or `--work-queue` flag: `prefect agent
start -q kubernetes`
Starting v2.3.2 agent connected to
<https://api.prefect.cloud/api/accounts/redact/work>
spaces/redact...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): kubernetes...
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/agent.py", line 88, in get_work_queues
work_queue = await self.client.create_work_queue(name=name)
File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 835, in create_work_queue
response = await <http://self._client.post|self._client.post>("/work_queues/", json=data)
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1842, in post
return await self.request(
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 279, in send
response.raise_for_status()
File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 225, in raise_for_status
raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<http://google.com/work_queues/>'
For more information check: <https://httpstatuses.com/404>
Ilya Galperin
09/12/2022, 6:33 PMChristopher Boyd
09/12/2022, 6:34 PMIlya Galperin
09/12/2022, 6:34 PMService
manifest here have any impact on connectivity between Cloud and the Agent? My understanding is that this would only be for communication between pods spawned by the agent and the agent itself, right?Starting v2.3.0 agent connected to <http://google.com>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): main...
Christopher Boyd
09/12/2022, 6:39 PMIlya Galperin
09/12/2022, 6:39 PMChristopher Boyd
09/12/2022, 6:40 PMAgents now support multiple work queues. Instead of passing a single argument,
provide work queue names with the `-q` or `--work-queue` flag: `prefect agent
start -q kubernetes`
Starting v2.3.2 agent connected to <https://google.com>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): kubernetes...
prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://google.com/work_queues/>'
For more information check: <https://httpstatuses.com/404>
Ilya Galperin
09/12/2022, 6:43 PMChristopher Boyd
09/12/2022, 6:44 PMIlya Galperin
09/12/2022, 6:45 PMChristopher Boyd
09/12/2022, 6:45 PMIlya Galperin
09/12/2022, 6:45 PMChristopher Boyd
09/12/2022, 6:51 PMIlya Galperin
09/12/2022, 6:54 PMChristopher Boyd
09/12/2022, 6:55 PMprefect cloud login --key <key>
Ilya Galperin
09/12/2022, 7:18 PM# apt-get install curl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package curl
# apt-get update
Err:1 <http://deb.debian.org/debian> bullseye InRelease
Temporary failure resolving '<http://deb.debian.org|deb.debian.org>'
Christopher Boyd
09/12/2022, 7:22 PMcat DOCKERFILE
FROM prefecthq/prefect:2.32.0-python3.10
RUN pip install httpie
then:
export IMAGE_REGISTRY="my_image_registry"
export PROJECT_NAME="prefect-httpie-3.10"
export PROJECT_VERSION="latest"
echo "$IMAGE_REGISTRY/$PROJECT_NAME:$PROJECT_VERSION"
docker build --platform=linux/amd64 -t "$IMAGE_REGISTRY/$PROJECT_NAME:$PROJECT_VERSION" -f ./DOCKERFILE .
docker push "$IMAGE_REGISTRY/$PROJECT_NAME:$PROJECT_VERSION"
Ilya Galperin
09/12/2022, 7:23 PMChristopher Boyd
09/12/2022, 7:24 PMhttp <https://api.prefect.io>
HTTP/1.1 400 Bad Request
Access-Control-Allow-Origin: *
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Content-Length: 18
Content-Type: text/html; charset=utf-8
Date: Mon, 12 Sep 2022 19:24:13 GMT
ETag: W/"12-7JEJwpG8g89ii7CR/6hhfN27Q+k"
Set-Cookie: route=1663010654.252.42.320963|0e12785838f6d88c8042f8a55371e727; Expires=Wed, 14-Sep-22 19:24:13 GMT; Max-Age=172800; Path=/; Secure; HttpOnly
Strict-Transport-Security: max-age=15724800; includeSubDomains
Via: 1.1 google
X-Powered-By: Express
GET query missing.
Ilya Galperin
09/12/2022, 7:25 PMChristopher Boyd
09/12/2022, 7:25 PMIlya Galperin
09/12/2022, 7:25 PM