https://prefect.io logo
Join Slack
Powered by
Hello, what would be the best way to pass a bitbuc...
# prefect-server
r
Hello, what would be the best way to pass a bitbucket access token to prefect-sever? (I.e. Set a secret in prefect-server) I setup the server using the instructions in the most recent

stream▾

When trying the following for a given flow
Copy code
flow.run_configs = KubernetesRun(
    image=<image>,
    env={
        "PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN": os.environ[
            "BITBUCKET_ACCESS_TOKEN"
        ]
    },
    image_pull_policy="Always",
)
flow.storage = Bitbucket(
    project=<project>,
    repo=<repo>,
    path="flows/example_flow.py",
    access_token_secret="BITBUCKET_ACCESS_TOKEN",
)
I get the following error on the UI
Copy code
Failed to load and execute Flow's environment: ValueError('Local Secret "BITBUCKET_ACCESS_TOKEN" was not found.')
Wouldn't the access token secret have been set in the run config? I also have the same environment variable in a custom docker image being used by the run config
k
Hey @Raed, Just confirming that registration works but just execution fails right?
So the environment variables passed from the RunConfig are applied after the Flow is loaded from storage. In this specific case, you’ll need to set it on the agent with 
prefect agent ____ start --env BITBUCKET_ACCESS_TOKEN=token
for the agent to be able to load the Flow from storage.
👍 1
r
Correct, registration worked but it fails on execution
Thanks I will try that, I also tried to set it in inside my values.yaml within the Perfect config but that did not work. To confirm, 'prefect agent kubernetes start' would automatically apply to my k8s cluster if I've entered the Apollo IP into my local config?
k
Yes the agent will hit the Apollo IP and be responsible for starting the k8s jobs.
r
So that would be equivalent to changing the values.yaml as follows?
Copy code
agent:
  # enabled determines if the Prefect Kubernetes agent is deployed
  enabled: true # So I dont have to manually start the agent and pass the variable 

  # prefectLabels defines what scheduling labels (not K8s labels) should
  # be associated with the agent
  prefectLabels: []

  # image configures the container image for the agent deployment
  image:
    repository: <Private Image> # imaage contains BITBUCKET_ACCESS_TOKEN as an environemnt variable 
    tag: latest
    pullPolicy: Always
    pullSecrets: []
So if the image is being pulled after the flow is loaded, I could just use a universal executor given my values.yaml contains all the needed params?
Copy code
flow.run_configs = KubernetesRun(
    image="<Private Image>:latest",
)
Also, if
Copy code
agent:
   enabled: true
does that mean running
Copy code
prefect agent ____ start --env BITBUCKET_ACCESS_TOKEN=token
will be ignored? Because kubernetes already has an agent service?
k
Hey @Raed, I talked to some team members about this. Can you try the original code but use 
flow.run_config
instead of 
flow.run_configs
?
r
Hey @Kevin Kho, good catch, I switched it to run_config but still get the same error, I figure this must be because of what you stated above regarding RunConfig being applied afterwards?
Copy code
[9 June 2021 4:44pm]: Failed to load and execute Flow's environment: ValueError('Local Secret "BITBUCKET_ACCESS_TOKEN" was not found.')
Any word on if the above values.yaml approach is alright? It'd be ideal to just have everything managed from within the helm chart, and would the above make the KubernetesRun object redundant?
k
So when I chatted with the engineers, the conclusion I got (which is different from I told you earlier) is that the RunConfig env variables will be enough to pull down the Flow from Storage. If you want to do it from Kubernetes, that contains that environment variable 
PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN
, then yes you don’t need to start a new one with the command line
Sorry, you’ll want 
BITBUCKET_ACCESS_TOKEN
in the environment instead of the other one.
r
So I created the env variable 
BITBUCKET_ACCESS_TOKEN
created an interactive shell onto the kubernetes agent, verified the key was correct with inside the pod
Copy code
echo $BITBUCKET_ACCESS_TOKEN
but still get the same error
Copy code
[9 June 2021 5:01pm]: Failed to load and execute Flow's environment: ValueError('Local Secret "BITBUCKET_ACCESS_TOKEN" was not found.')
So you're saying I should change the name of the env variable from 
BITBUCKET_ACCESS_TOKEN
to 
PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN
? In both
Copy code
flow.run_config = KubernetesRun(
    image=<private docker image>,
    env={"PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN": os.environ["KS_BITBUCKET_ACCESS_TOKEN"]},
)
& the docker image?
k
Just on the values.yaml should be good. We shouldn’t need the env anymore in KubernetesRun. Yes what you tried is what I was suggesting.
Checking more things
Do you only have one agent on? Just wanna make sure the right agent is picking up the Flow
r
Correct, only one agent
k
Can you try removing the access token secret from the Bitbucket storage like:
Copy code
flow.run_config = KubernetesRun(
    image=<image>,
    env={
        "BITBUCKET_ACCESS_TOKEN": os.environ[
            "BITBUCKET_ACCESS_TOKEN"
        ]
    },
    image_pull_policy="Always",
)
flow.storage = Bitbucket(
    project=<project>,
    repo=<repo>,
    path="flows/example_flow.py",
)
r
Tried that just now, authorization failed
Copy code
[9 June 2021 5:22pm]: Failed to load and execute Flow's environment: HTTPError('401 Client Error: Unauthorized for url:<URL>')
Also building an image with 
PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN
just in case, read somewhere that it expects all secrets in that format and would make sense why it's failing to find env
k
Are you trying to hit Bitbucket Cloud or Server?
r
Bitbucket Cloud
Will report back later on if putting the below in the image worked 
PREFECT__CONTEXT__SECRETS__BITBUCKET_ACCESS_TOKEN
k
It looks like the env variable was picked up but the token is unauthorized on Bitbucket.
I’m looking at the documentation and it looks like the ACCESS_TOKEN_SECRET was for Bitbucket Server only. Cloud uses username and password for authentication. Looking at the code if you can use the token with Cloud
My bad. I should have asked if Bitbucket Cloud or Server earlier. Cloud requires username and password to authenticate here
You need 
BITBUCKET_CLOUD_USERNAME
and 
BITBUCKET_CLOUD_APP_PASSWORD
as the environment variables for this
r
Copy code
It looks like the env variable was picked up but the token is unauthorized on Bitbucket.
The token isn't picked up at all, have used this token in the pass successfully and I believe the error message above says it doesn't exist.
Thanks for catching that, I will try adding those as variables in my image and reporting back
k
The error is happening because the code directs 
BITBUCKET_ACCESS_TOKENS
to Bitbucket Server, which causes the “Unauthorized” error, but you’re using Bitbucket Cloud. The token probably works for cloud but the way Bitbucket Storage is coded directs Access Tokens to server and username + password to Cloud.
👍 1
Do you have a workspace for Cloud you can pass to the Storage?
Am reading the code again and it uses Prefect Cloud if you have a workspace, but uses Prefect Server if you don’t have one and I didnt see a workspace in the codesnippet earlier.
r
Yes I have a workspace, I've updated the code as follows
Copy code
flow.run_config = KubernetesRun(
    image=<private_image>,
    env={
        "BITBUCKET_CLOUD_USERNAME": os.environ["BITBUCKET_CLOUD_USERNAME"],
        "BITBUCKET_CLOUD_APP_PASSWORD": os.environ["BITBUCKET_ACCESS_TOKEN"],
    },
   


# * bitbucket storage
flow.storage = Bitbucket(
    project=<project>,
    repo=<repo>,
    workspace=<workspace>,
    cloud_username_secret="BITBUCKET_CLOUD_USERNAME",
    cloud_app_password_secret="BITBUCKET_CLOUD_APP_PASSWORD",
    path="flows/flow.py",
)
Gets error
Copy code
[10 June 2021 1:26pm]: Failed to load and execute Flow's environment: ValueError('Local Secret "BITBUCKET_CLOUD_USERNAME" was not found.')
Do you think that might have to do with formatting of environment variables so they can be used as secrets? That is
Copy code
flow.run_config = KubernetesRun(
    image=<private_image>,
    env={
        "PREFECT__CONTEXT__SECRETS__BITBUCKET_CLOUD_USERNAME": os.environ["BITBUCKET_CLOUD_USERNAME"],
        "PREFECT__CONTEXT__SECRETS__BITBUCKET_CLOUD_APP_PASSWORD": os.environ["BITBUCKET_ACCESS_TOKEN"],
    },
I think this may have worked, am getting a new error when using the new naming convention, assuming it must not be able to find the flow in the repo, is there an easy way to see the full stacktrace on the UI?
Copy code
[10 June 2021 1:32pm]: Failed to load and execute Flow's environment: FileNotFoundError(2, 'No such file or directory')
k
Was gonna say I think the second one will work with, bPREFECT__CONTEXT__SECRETS__ but was verifying. Do you have DEBUG level logs enabled? I think you’ll get more logs if you do that. This specific error means than it can’t find the flow at the place you pointed in Bitbucket. Did you supply a path?
The path here is where it will look for the flow to download.
Copy code
flow = Flow("my-flow")
flow.storage = Bitbucket(
    project="my.project", repo="my.repo", path="/flows/flow.py", ref="my-branch"
)
What version of Prefect are you on also?
r
Copy code
prefect -- 0.14.21
k
0.14.19 had issues with CLI registration but you should be fine. We just need to provide the path to the file in Bitbucket
r
Seems like it tries in downloading the flow, but as soon as it begins it throws this error
Copy code
Failed to load and execute Flow's environment: KeyError('BITBUCKET_CLOUD_USERNAME')
So now the actual environment variable is missingm but that's odd because my flow originally had this line upon registration
Copy code
flow.run_config = KubernetesRun(
    image=<image>,
    env={
        "PREFECT__CONTEXT__SECRETS__BITBUCKET_CLOUD_USERNAME": os.environ[
            "BITBUCKET_CLOUD_USERNAME"
        ],
        "PREFECT__CONTEXT__SECRETS__BITBUCKET_CLOUD_APP_PASSWORD": os.environ[
            "BITBUCKET_ACCESS_TOKEN"
        ],
        "PREFECT__LOGGING__LEVEL": "DEBUG",
    },
    image_pull_policy="IfNotPresent",
)
When the flow runs I presume this must cause that error by trying to run os.environ['BITBUCKET_CLOUD_USERNAME'], but passing debug doesn't provide the full stack trace, any idea how I can get it? Also shouldn't the above line automatically adjust the agent service on kubernetes to have those environment variables? Or do I have to restart the pod manually ?
k
It actually says 
KS_BITBUCKET_CLOUD_USERNAME
? That’s so weird. Let me chat with the team about full traceback.
r
sorry should say "BITBUCKET_CLOUD_USERNAME"
k
Can I see how you set up the RunConfig? Are you doing it inside the 
with Flow() as flow:
block or outside?
r
Outside, after the context manager
k
Ok that should be good. Will ask the team about debugging.
I’ll try to get a working example for Bitbucket today I think I have some time
r
that would be awesome, thanks again for all your help! I suspect the issue is what you mentioned in the beginning is whats causing this error
Copy code
So the environment variables passed from the RunConfig are applied after the Flow is loaded from storage.
So the flow is loaded, the run config is executed, but the env variables are not there so it fails, trying again with a rebuilt image that has the correct env vars
k
I was wrong when I said that. I have a working script with Bitbucket. I will DM you because I have a password written out.
10 Views
Open in Slack
PreviousNext
Powered by