Ben Welsh
01/27/2022, 6:17 PMKevin Kho
01/27/2022, 7:05 PMBen Welsh
01/27/2022, 7:24 PMFROM prefecthq/prefect:latest
COPY dockerfile-requirements.txt /requirements.txt
RUN pip install -r requirements.txt
Kevin Kho
01/27/2022, 7:28 PMprefect agent kubernetes install
, it generates a spec. Look for the image line in that spec and replace it with yours. Then you need to push your image somewhere where it can be pulled by kubernetes.Ben Welsh
01/27/2022, 8:03 PMKevin Kho
01/27/2022, 8:11 PMprefect agent local start
. You can spin it up in a VM the same way.Ben Welsh
01/27/2022, 8:23 PMprefect agent kubernetes install -k $API_KEY > k8s.yml
kubectl apply -f k8s.yml
Kevin Kho
01/27/2022, 8:25 PMBen Welsh
01/27/2022, 8:25 PMKevin Kho
01/27/2022, 8:31 PMBen Welsh
01/27/2022, 8:32 PMKevin Kho
01/27/2022, 8:34 PMBen Welsh
01/27/2022, 8:34 PMraise ClientError("Malformed response received from API.") from exc prefect.exceptions.ClientError: Malformed response received from API.
Kevin Kho
01/27/2022, 8:34 PMBen Welsh
01/27/2022, 8:36 PMpipenv run prefect agent local start -k $API_KEY;
$ pipenv run prefect agent local start -k $API_KEY;
Loading .env environment variables...
[2022-01-27 20:35:53,744] INFO - agent | Registering agent...
[2022-01-27 20:35:53,777] INFO - agent | Registration successful!
__ __ _ _ _
| _ \ _ __ _ / _| _ ___| |_ / \ __ _ _ _ _ | |
| |_) | '__/ _ \ |_ / _ \/ __| __| / _ \ / _` |/ _ \ '_ \| __|
| __/| | | __/ _| __/ (_| | / _ \ (_| | _/ | | | |
|_| |_| \___|_| \___|\___|\__| /_/ \_\__, |\___|_| |_|\__|
|___/
[2022-01-27 20:35:53,786] INFO - agent | Starting LocalAgent with labels ['little-tokyo']
[2022-01-27 20:35:53,786] INFO - agent | Agent documentation can be found at https://docs.prefect.io/orchestration/
[2022-01-27 20:35:53,786] INFO - agent | Waiting for flow runs...
[2022-01-27 20:35:53,829] INFO - agent | Deploying flow run 7afc4aaf-28e2-474b-b62d-6894458341c4 to execution environment...
[2022-01-27 20:35:53,876] INFO - agent | Completed deployment of flow run 7afc4aaf-28e2-474b-b62d-6894458341c4
Kevin Kho
01/27/2022, 8:38 PMBen Welsh
01/27/2022, 8:40 PM- name: PREFECT__BACKEND
value: server
Kevin Kho
01/27/2022, 8:44 PMBen Welsh
01/27/2022, 8:45 PM- name: PREFECT__CLOUD__AGENT__LABELS
value: '["warn-prefect-flow"]'
Kevin Kho
01/27/2022, 8:56 PMBen Welsh
01/27/2022, 8:57 PMState Message:
(403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': 'a4cc413f-ee09-419c-a727-5d307c0f199d', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': 'b72eaaa4-778e-43f7-9d3e-16334e3b1250', 'X-Kubernetes-Pf-Prioritylevel-Uid': '11303276-a26e-4d06-b683-8b59c0220e2f', 'Date': 'Thu, 27 Jan 2022 21:01:54 GMT', 'Content-Length': '311'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system😒erviceaccount:default:default\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"default\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403}
Kevin Kho
01/27/2022, 9:10 PMBen Welsh
01/27/2022, 9:13 PMFailed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/palewire/Code/warn-prefect-flow/flow'")
FROM prefecthq/prefect:latest
COPY dockerfile-requirements.txt /requirements.txt
RUN pip install -r requirements.txt
COPY flow.py /flow.py
Kevin Kho
01/27/2022, 9:21 PMBen Welsh
01/27/2022, 9:22 PMKevin Kho
01/27/2022, 9:23 PMBen Welsh
01/27/2022, 9:27 PMBen Welsh
01/27/2022, 9:33 PMKevin Kho
01/27/2022, 9:36 PMBen Welsh
01/27/2022, 9:39 PMKevin Kho
01/27/2022, 9:52 PMBen Welsh
01/27/2022, 9:54 PMKevin Kho
01/27/2022, 9:56 PMBen Welsh
01/27/2022, 9:58 PMKevin Kho
01/27/2022, 10:01 PM