https://prefect.io logo
s

Sai Srikanth

01/20/2021, 10:11 PM
Hello all, I am trying to connect Prefect, AWS thru storing secrets as ACCESS_KEY, SECRET_KEY, SECURITY_TOKEN. I am trying to connect as below but I am unable to connect. Any suggestions? Please check the image for further reference:
m

Mariia Kerimova

01/20/2021, 10:40 PM
Hello @Sai Srikanth! You're passing secrets to task.run() directly which won’t resolve them into their values. You can try following:
Copy code
@task(...)
def InvokeLambda(creds):
    pass

with Flow("test") as flow:
    access_key = PrefectSecret("ACCESS_KEY")
    ...
s

Sai Srikanth

01/20/2021, 10:45 PM
Hi Maria, Thanks for the update. Please find below, is this the best way to use the secrets? in that case how do we assign tasks for the flow?
m

Mariia Kerimova

01/20/2021, 10:56 PM
in this case you can simply call
InvokeLambdaFunction(creds)
s

Sai Srikanth

01/20/2021, 11:05 PM
Thanks @Mariia Kerimova , the run is now successful, but I am not getting the list of lambda functions in the prefect console/ local. How should I check that? Attached the logs extracted from prefect
m

Mariia Kerimova

01/21/2021, 12:06 AM
Hey Sai!
Copy code
from prefect.tasks.secrets import PrefectSecret
import prefect
from prefect import Flow, task
from prefect.tasks.aws.lambda_function import LambdaList

lambda_list = LambdaList(master_region="ap-southeast-2", function_version="ALL")

@task()
def print_result(result):
    <http://prefect.context.logger.info|prefect.context.logger.info>(result['Functions'])


with Flow("test") as flow:
    access_key = PrefectSecret("ACCESS_KEY")
    secret_key = PrefectSecret("SECRET_ACCESS_KEY")
    creds = {"ACCESS_KEY": access_key, "SECRET_ACCESS_KEY": secret_key}
    result = lambda_list(creds)
    print_result(result)

flow.register(project_name="Test PoC")
But for local run you’ll need to set secrets as env variables(`PREFECT__CONTEXT__SECRETS__ACCESS_KEY`and
PREFECT__CONTEXT__SECRETS__SECRET_ACCESS_KEY
. See details here.
s

Sai Srikanth

01/21/2021, 12:12 AM
should we run PREFECT__CONTEXT__SECRETS__ACCESS_KEY in CLI/ Python?
m

Mariia Kerimova

01/21/2021, 12:17 AM
In case if you want run locally(
flow.run()
), export env variables in CLI, then run flow
s

Sai Srikanth

01/21/2021, 12:21 AM
The below is the results as per the run gives no functions, but when I run the same through CLI, I get list of functions
👀 1
3 Views