c

    Charles Leung

    1 year ago
    Hello Prefect Team, I'm trying to use local secrets with the core server setup to test some things. However, I constantly get an key error even as i set the environment variable via the agent as such: prefect agent start docker --env PREFECT__CONTEXT__SECRETS__VAULT_TOKEN=<REDACT> --api <REDACT> Unexpected error: HTTPError('400 Client Error: Bad Request for url: <REDACT>/graphql') Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/prefect/client/secrets.py", line 137, in get value = secrets[self.name] File "/usr/local/lib/python3.7/site-packages/box/box.py", line 335, in getitem raise BoxKeyError(str(err)) from None box.exceptions.BoxKeyError: "'vault_token'" Am i missing something in my setup? use_local_secrets is set to true in the config.
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    It looks like you're setting your key as
    VAULT_TOKEN
    , but requesting it as
    vault_token
    (keys are case sensitive). I'd start trying with matching case and see if that works.
    c

    Charles Leung

    1 year ago
    I actually tried that first, and got the same issue 😞
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    What version of prefect is your agent running?
    c

    Charles Leung

    1 year ago
    0.13.5
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Ah, that's very old. Several bugs when using secrets with prefect server have been fixed since then, I'd suggest upgrading to see if that fixes the issue.
    c

    Charles Leung

    1 year ago
    ok got it, thanks!
    Hey Jim, Thanks for your quick response. I've updated my version and i now get the following error:
    Unexpected error: ValueError('Local Secret "VAULT_TOKEN" was not found.')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
        new_state = method(self, state, *args, **kwargs)
      File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 869, in get_task_run_state
        logger=self.logger,
      File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 299, in run_task_with_timeout
        return task.run(*args, **kwargs)  # type: ignore
      File "/usr/local/lib/python3.7/site-packages/prefect/tasks/secrets/base.py", line 68, in run
        return _Secret(name).get()
      File "/usr/local/lib/python3.7/site-packages/prefect/client/secrets.py", line 142, in get
        ) from None
    ValueError: Local Secret "VAULT_TOKEN" was not found.
    Is there a way to debug inside the environment?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    How are you accessing the secret? With a
    PrefectSecret
    object?
    c

    Charles Leung

    1 year ago
    yes
    import os
    import prefect
    from prefect.tasks.secrets import PrefectSecret
    from prefect.environments.storage.docker import Docker
    
    flow = prefect.Flow('vault-test')
    fetch_token = PrefectSecret('VAULT_TOKEN')
    flow.add_task(fetch_token)
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Hmmm. Try registering and running the following flow and see what it outputs (with the same agent configuration):
    import os
    import prefect
    from prefect import Flow, task
    
    
    @task(log_stdout=True)
    def log_env():
        print("Logging prefect environment variables")
        for k, v in os.environ.items():
            if k.startswith("PREFECT_"):
                print(f"ENVAR: {k} = {v}")
    
    
    @task(log_stdout=True)
    def log_secrets():
        print("Logging prefect secrets")
        for k in prefect.context.get("secrets", {}):
            print(f"SECRET: {k}")
    
    
    with Flow("debug") as flow:
        log_env()
        log_secrets()
    c

    Charles Leung

    1 year ago
    Perfect Jim 🙂 this task helped me find what i needed. I was using jinja templating to create a systemctl prefect agent service, and a \n between the --env and the initial command was breaking it. Thanks!
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Glad you figured it out!