Trying to get k8s worker in AWS EKS Cluster connected to Prefect Cloud Account. • I’ve followed the docs here for using helm chart for prefect workers • Only difference is we’re using weave-gitops instead of installing the helm chart manually • I’ve added my API key (no expiration) and base64 encoded it, added as secret and ref’d the secret in helm chart • Ensured that the cloudApiConfig is filled out (i.e accountId , workspaceId) • Ensured that the workPool is filled out with workpool name ◦ I tried leaving this empty and then also specifying a k8s work pool I created in the ui • Wondering if my cloud url is proper ? I’ve tried 127.0.0.1 / 0.0.0.0 / and https://api.prefect.cloud/api ◦

apiUrl: <http://127.0.0.1:4200/api>

- this is default in the helm chart and docs don’t mention changing this Error I receive when looking at the logs of the worker

Worker 'KubernetesWorker 3f629027-a4e8-4b08-b856-2e5c62e38933' started!
15:01:39.075 | DEBUG   | prefect.worker.kubernetes.kubernetesworker 3f629027-a4e8-4b08-b856-2e5c62e38933 - Exiting worker context...
15:01:39.075 | DEBUG   | prefect.worker.kubernetes.kubernetesworker 3f629027-a4e8-4b08-b856-2e5c62e38933 - Tearing down worker...
15:01:39.076 | DEBUG   | MainThread   | prefect._internal.concurrency - <AsyncCancelScope, name='start' COMPLETED, runtime=2.02> exited
15:01:39.076 | DEBUG   | MainThread   | prefect._internal.concurrency - Encountered exception in async call start(worker_name=None, work_pool_name='jey-test-custom-k8s-work-pool', work_queues=[], worker_type='kuber...)
Traceback (most recent call last):

..... shortened logs.....

prefect.exceptions.PrefectHTTPStatusError: Client error '403 Forbidden' for url '<https://api.prefect.cloud/api/accounts/><REMOVED_ACCOUNT_ID>/workspaces/<REMOVED_WORKSPACE_ID>/work_pools/jey-test-custom-k8s-work-pool'
Response: {'detail': 'Unauthorized'}
For more information check: <https://httpstatuses.com/403>

If you’re trying to connect to your cloud account , it should be a secret + key reference to an existing secret , as well as the prefect api url - I can share an example momentarily

within cloudApiConfig I’ve got

apiKeySecret:
          # -- prefect API secret name
          name: prefect-api-key
          # -- prefect API secret key
          key: key

referencing a key I’ve created manually within the prefect namespace

k get secret -n prefect
NAME                                   TYPE                 DATA   AGE
prefect-api-key                        Opaque               1      3d21h

that should be right, assuming your key is in fact key (so like

k create secret generic prefect-api-key --from-literal 'key=<actual_key>

I’ve decrypted it and ensured it’s the proper b64 encoded key. Unsure if this is the issue, but I’m not doing a local helm install, it’s all through weave-gitops which would look at the yaml configs and do the helm install itself

You can check that your api key + url are valid - https://discourse.prefect.io/t/how-can-i-tell-if-there-is-an-issue-with-my-api-key/2185 The 403 in particular suggests to me that the URL is probably OK (because you aren’t getting a 404) but either the wrong api key (that doesn’t have permissions to that workspace) or an incorrect api key in general

So I’ve ran that curl and get a 403. I use the same base64 decoded key with

prefect cloud login

command and I’m able to login 🤔 I am on the free plan for the POC does that matter?

what does prefect profile inspect show when you’re logged in

It shows the key and then an API url . I see that the account ID in the API url doesn’t match my account id that shows up in the UI..

If you logged in via cli

prefect cloud login -k <key>

then selected a workspace, that should be what prefect profile inspect logged you into with that key (and by extension where that key was generated). If you’re in the UI with a different account, you should be able to just generate a new api key and use that

The URL account ID matches the cli command you provided. Well that’s frustrating

I used the same instructions and got same error with the API Key! Was there a resolution for that?

I had to use the account ID in the URL as that’s what is tied to the namespace

Thanks, it also resolved my problem. It is strange because the account id on My Profile on the UI is different than the account id on the URL when I enter the workspace on the UI.

Yeah the docs are a bit misleading . They put in an MR to revise the docs to tell folks to pull from the URL instead. Imo user account id / workspace account id should probably be named a bit differently.