Zhibin Dai

    Zhibin Dai

    7 months ago
    Hi All! My team just started using Prefect and we are facing some growing pains. I have a flow that runs fine locally, but when I register it and run it through a Local Agent, the Agent is able to pick it up, but the flow is failing because it's not reading secrets stored in Prefect Cloud, even though the secrets are there. Please see screenshots. Any help is appreciated, Thanks!
    Kevin Kho

    Kevin Kho

    7 months ago
    Hi @Zhibin Dai, could you show me the code that causes this?
    Can you try doing this on the box with the agent:
    from prefect.client import Secret
    Secret("GITHUB_ACCESS_TOKEN").get()
    and then print that to see if you can retrieve it?
    Zhibin Dai

    Zhibin Dai

    7 months ago
    yeah ill try that
    FLOW_NAME = "02_dbt_staging"
    GITHUB_ACCESS_TOKEN = PrefectSecret('GITHUB_ACCESS_TOKEN')
    STORAGE = GitHub(
    	repo="",
    	path=f"flows/ppi/{FLOW_NAME}.py",
    	access_token_secret=GITHUB_ACCESS_TOKEN
    )
    this is the code, i removed the repo here but its in the real code
    Kevin Kho

    Kevin Kho

    7 months ago
    Ah ok
    PrefectSecret('GITHUB_ACCESS_TOKEN')
    is evaluated during the Flow run because it is a task so you should only use it inside Flows. You can use
    Secret.get()
    here instead
    Zhibin Dai

    Zhibin Dai

    7 months ago
    When i use secret.get() its saying the literal gh token string is not found
    └── 20:41:03 | ERROR   | Failed to load and execute Flow's environment: KeyError('The secret ghp_R... was not found.  Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>')
    GITHUB_ACCESS_TOKEN = Secret("GITHUB_ACCESS_TOKEN").get()
    STORAGE = GitHub(
    	repo="",
    	path=f"flows/ppi/{FLOW_NAME}.py",
    	access_token_secret=GITHUB_ACCESS_TOKEN
    )
    Kevin Kho

    Kevin Kho

    7 months ago
    Oh sorry you don’t even need that.
    access_token_secret
    takes in a string
    access_token_secret="GITHUB_ACCESS_TOKEN"
    and then Prefect will fetch the secret for you. Just take out the line that uses the String
    Zhibin Dai

    Zhibin Dai

    7 months ago
    the keyerror is gone but now im getting 404
    Failed to load and execute Flow's environment: UnknownObjectException(404, {'message': 'Not Found', '
    Kevin Kho

    Kevin Kho

    7 months ago
    That’s better. Is it right that repo is
    ""
    or you took it out just to paste the code? What is
    FLOW_NAME
    in that script?
    Zhibin Dai

    Zhibin Dai

    7 months ago
    FLOW_NAME = "02_dbt_staging"
    the repo is populated
    Kevin Kho

    Kevin Kho

    7 months ago
    Could you DM me your script just so I can see if everything looks good?
    It actually looks good man
    At this point I can only think your token doesn’t have permissions to that path or the file really doesn’t exist
    Zhibin Dai

    Zhibin Dai

    7 months ago
    i think its a branch issue now
    how do i specify a feature branch?
    Kevin Kho

    Kevin Kho

    7 months ago
    I think there is a
    ref
    keyword in Github storage
    Zhibin Dai

    Zhibin Dai

    7 months ago
    ah yes
    its working now, thank you!
    Kevin Kho

    Kevin Kho

    7 months ago
    Of course!