Robert Bastian
12/15/2020, 3:08 PMdef get_record_count():
logger = prefect.context.get("logger")
pg = PostgresFetch(host='<http://hostname.us-east-1.rds.amazonaws.com|hostname.us-east-1.rds.amazonaws.com>', db_name='testdatabase', user='test')
pg.query="select count(*) from testdatabase.test_table"
secret = PrefectSecret('TEST_PASSWORD')
password = secret.run()
result = pg.run(password=password)
return result[0]
Jim Crist-Harif
12/15/2020, 3:19 PMrun
method (as you've done above). An alternate way would be to have two tasks - one for getting the secret, and one for running the query. This lets you compose things from the task library better without having to call run
yourself. Something like the following (untested):
with Flow("your-flow") as flow:
password = PrefectSecret("TEST_PASSWORD")
pg = PostgresFetch(
host='<http://hostname.us-east-1.rds.amazonaws.com|hostname.us-east-1.rds.amazonaws.com>',
db_name='testdatabase',
user='test',
query="select count(*) from testdatabase.test_table"
)
result = pg(password=password)
Robert Bastian
12/15/2020, 6:51 PMJim Crist-Harif
12/15/2020, 6:59 PMrun
method is a called at flow runtime. In your first example, you wrote a function that manually created to Task
objects (but didn't add them to a flow), then called their run
methods manually. In this case, they're more like helper functions you're using to do your work.
The former (compose tasks into flows) is the recommended way to use prefect, but the latter (use parts of our task library to build your own tasks) also works.