Sai Srikanth

    Sai Srikanth

    1 year ago
    Hello all, I am trying to connect Prefect, AWS thru storing secrets as ACCESS_KEY, SECRET_KEY, SECURITY_TOKEN. I am trying to connect as below but I am unable to connect. Any suggestions? Please check the image for further reference:
    m

    Mariia Kerimova

    1 year ago
    Hello @Sai Srikanth! You're passing secrets to task.run() directly which won’t resolve them into their values. You can try following:
    @task(...)
    def InvokeLambda(creds):
        pass
    
    with Flow("test") as flow:
        access_key = PrefectSecret("ACCESS_KEY")
        ...
    Sai Srikanth

    Sai Srikanth

    1 year ago
    Hi Maria, Thanks for the update. Please find below, is this the best way to use the secrets? in that case how do we assign tasks for the flow?
    m

    Mariia Kerimova

    1 year ago
    in this case you can simply call
    InvokeLambdaFunction(creds)
    Sai Srikanth

    Sai Srikanth

    1 year ago
    Thanks @Mariia Kerimova , the run is now successful, but I am not getting the list of lambda functions in the prefect console/ local. How should I check that? Attached the logs extracted from prefect
    m

    Mariia Kerimova

    1 year ago
    Hey Sai!
    from prefect.tasks.secrets import PrefectSecret
    import prefect
    from prefect import Flow, task
    from prefect.tasks.aws.lambda_function import LambdaList
    
    lambda_list = LambdaList(master_region="ap-southeast-2", function_version="ALL")
    
    @task()
    def print_result(result):
        <http://prefect.context.logger.info|prefect.context.logger.info>(result['Functions'])
    
    
    with Flow("test") as flow:
        access_key = PrefectSecret("ACCESS_KEY")
        secret_key = PrefectSecret("SECRET_ACCESS_KEY")
        creds = {"ACCESS_KEY": access_key, "SECRET_ACCESS_KEY": secret_key}
        result = lambda_list(creds)
        print_result(result)
    
    flow.register(project_name="Test PoC")
    But for local run you’ll need to set secrets as env variables(PREFECT__CONTEXT__SECRETS__ACCESS_KEYand
    PREFECT__CONTEXT__SECRETS__SECRET_ACCESS_KEY
    . See details here.
    Sai Srikanth

    Sai Srikanth

    1 year ago
    should we run PREFECT__CONTEXT__SECRETS__ACCESS_KEY in CLI/ Python?
    m

    Mariia Kerimova

    1 year ago
    In case if you want run locally(
    flow.run()
    ), export env variables in CLI, then run flow
    Sai Srikanth

    Sai Srikanth

    1 year ago
    The below is the results as per the run gives no functions, but when I run the same through CLI, I get list of functions