Jason
05/10/2022, 7:33 PMname
of a SnowflakeQuery
task similar to @task to give it a human meaning name? I didn't see it in https://docs.prefect.io/api/latest/tasks/snowflake.htmlAndrew Huang
05/10/2022, 7:42 PMname='human_meaning_name'
to the **kwargs
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor
Jason
05/10/2022, 7:43 PMAndrew Huang
05/10/2022, 7:44 PMJason
05/10/2022, 7:48 PMmap
with SnowflakeQuery
to load multiple parquet datasets? Or are these usually discrete tasks?Andrew Huang
05/10/2022, 7:57 PMsnowflake_query = SnowflakeQuery(
account=Secret("SNOWFLAKE_ACCOUNT").get(),
user=Secret("SNOWFLAKE_USER").get(),
password=Secret("SNOWFLAKE_PASSWORD").get(),
...
)
with Flow("weather_notifications", schedule=schedule) as flow:
snowflake_query.map(...)
Kevin Kho
05/10/2022, 8:28 PMJason
05/10/2022, 8:46 PM@task
def snowflake_query(name, s3_path):
return SnowflakeQuery(...)
Kevin Kho
05/10/2022, 8:48 PMwith Flow():
snowflake_query.map(query=list_of_queries)
For exampleJason
05/10/2022, 8:49 PM