Thread
#prefect-community
    b

    Brad I

    1 year ago
    I also had an issue with the k8s agent and the new API key. From the new docs it seemed like you should just remove or blank out the old AUTH_TOKEN env var, but when I created a new service account key and deployed it, I get a message like
    failed to authenticate, missing token
    . It works if I set the key in both variables, is this expected?
    env:
    - name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
      value: XXXXXXXX
    - name: PREFECT__CLOUD__API
      value: <https://api.prefect.io>
    - name: PREFECT__BACKEND
      value: cloud
    - name: PREFECT__CLOUD__API_KEY
      value: XXXXXXXX
    - name: PREFECT__CLOUD__TENANT_ID
      value: TTTTTTTT
    image: prefecthq/prefect:0.15.0-python3.7
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Brad I, I’ll ask the team about this
    Michael Adkins

    Michael Adkins

    1 year ago
    Hey @Brad I -- could you give me the exact error message you saw? This should work but perhaps there's something weird about the K8s agent specifically.
    b

    Brad I

    1 year ago
    Hi @Michael Adkins, I was also getting our infra setup using Pulumi, so I’ve lost the full log but the error message was:
    Failed to verify authentication
    Tyler Wanner

    Tyler Wanner

    1 year ago
    Hi Brad! Can you verify for me that if the auth token var is unset entirely, it works as expected? And that the error you’re seeing is the agent talking to Cloud directly
    Michael Adkins

    Michael Adkins

    1 year ago
    The code that would produce that is...
    # Verify API tokens -- API keys do not need a type-check
            if config.backend == "cloud" and not self.client.api_key:
                self.logger.debug("Verifying authentication with Prefect Cloud...")
                try:
                    self._verify_token(self.client.get_auth_token())
                    self.logger.debug("Authentication successful!")
                except Exception as exc:
                    self.logger.error("Failed to verify authentication.")
                    raise RuntimeError(
                        f"Error while contacting API at {config.cloud.api}",
                    ) from exc
    Which should only run if
    not self.client.api_key
    b

    Brad I

    1 year ago
    I tried it with the
    PREFECT__CLOUD__AGENT__AUTH_TOKEN
    env var completely removed and with it set to an empty string, received the same error message for both variations. That looks like the same code I found when googling the error message, I didn’t have to change my API_KEY var to make it work so I’m not sure why that block executed. I can try it again at some point today when I spin up our prod agent.
    Michael Adkins

    Michael Adkins

    1 year ago
    Very peculiar. We'll also try to reproduce over here.
    Well.. I can reproduce this. For some reason the environment variable isn't being read although I can load it easily if I spawn python separately
    Thanks for the report!
    b

    Brad I

    1 year ago
    Thanks @Michael Adkins!