Thread
#prefect-server
    Raúl Mansilla

    Raúl Mansilla

    1 year ago
    Hello everyone, I´m trying to use gitlab storage in one test-flow…I stored the gitlab token inside config.toml under
    [context.secrets]
    The flow seems to get the token as the error that raises the flow run in the UI says that the value was not found ->
    Failed to load and execute Flow's environment: ValueError('Local Secret "RZFXuBfesc_XXXXXXXXX" was not found.')
    Prefect and Gitlab are two ec2 instances and the sg of gitlab one allow ssh and https from prefect’ s ip.
    import prefect
    from prefect import task, Flow, Parameter
    from prefect.executors import LocalExecutor
    from prefect.run_configs import LocalRun
    from prefect.storage import GitLab
    import requests, os
    
    from prefect.client.secrets import Secret
    s = Secret("PREFECT__CONTEXT__SECRETS__GITLAB_ACCESS_TOKEN")
    
    @task()
    def overon_get_missing(days):
        url = "<https://XXXXXXXXX.execute-api.eu-west-1.amazonaws.com/prod/api/v1/getMissing/>"+ str(days)
        response = <http://requests.post|requests.post>(url)
        return response
    
    with Flow("overon_get_missing",run_config=LocalRun(),executor=LocalExecutor()) as flow:
        days = Parameter('days')
        result = overon_get_missing(days)
        print(result)
    
    flow.storage = GitLab(
        repo="dmed/soa/prefect/overon_get_missing_test",                            # name of repo
        path="overon_get_missing_gitlab.py",                    # location of flow file in repo
        access_token_secret=s.get(),   # name of personal access token secret
        host="<https://gitlab.test.net>",
        ref = "master"
    )
    
    overon_get_missing.run(days=30)
    nicholas

    nicholas

    1 year ago
    Hi @Raúl Mansilla - where is the
    config.toml
    file located from which you're trying to pull the secret?
    Raúl Mansilla

    Raúl Mansilla

    1 year ago
    Hello @nicholas,
    ./prefect/config.toml
    I mean, home directory of the main user…then the path I wrote…
    nicholas

    nicholas

    1 year ago
    It looks like you're passing in the secret value at registration time instead of the secret reference with s.get(); I believe gitlab storage is looking only for the reference
    GITLAB_ACCESS_TOKEN
    Raúl Mansilla

    Raúl Mansilla

    1 year ago
    I also tried doing an export and trying to get as GITLAB_ACCESS_TOKEN, but the same happens….what do you suggest?
    codingtacos

    codingtacos

    1 year ago
    Another option that I am using is AWS Secrets Manager. Its handy/cheap if you don't want to store the secrets on the server.
    import boto3 
    
    security_client = boto3.client('secretsmanager')
    security_responese = security_client.get_secret_value(SecretId='name_of_my_secret_in_aws')
    security_dict = json.loads(security_responese['SecretString'])
    
    print(security_dict['username'])
    Raúl Mansilla

    Raúl Mansilla

    1 year ago
    @codingtacos that is one thing I wanted to do 🙂 thx!
    codingtacos

    codingtacos

    1 year ago
    Nice! I use it on all my flows 🙂
    nicholas

    nicholas

    1 year ago
    Try this @Raúl Mansilla :
    flow.storage = GitLab(
        # ...
        access_token_secret='GITLAB_ACCESS_TOKEN',
        # ...
    )
    Raúl Mansilla

    Raúl Mansilla

    1 year ago
    mmm, O thought I tested it before, now it seems to be working better though now I have a timeout error trying to conect via https…I will check de sg rules. Thanks @nicholas