https://prefect.io logo
#prefect-community
Title
# prefect-community
j

Jason

05/10/2022, 7:33 PM
Is there a way to customize the
name
of a
SnowflakeQuery
task similar to @task to give it a human meaning name? I didn't see it in https://docs.prefect.io/api/latest/tasks/snowflake.html
a

Andrew Huang

05/10/2022, 7:42 PM
I think you can specify
name='human_meaning_name'
to the
**kwargs
Copy code
**kwargs (dict, optional): additional keyword arguments to pass to the Task constructor
j

Jason

05/10/2022, 7:43 PM
Ah, right. Thanks@!
a

Andrew Huang

05/10/2022, 7:44 PM
no problem!
j

Jason

05/10/2022, 7:48 PM
Has anyone used
map
with
SnowflakeQuery
to load multiple parquet datasets? Or are these usually discrete tasks?
a

Andrew Huang

05/10/2022, 7:57 PM
I think you can map it
Copy code
snowflake_query = SnowflakeQuery(
    account=Secret("SNOWFLAKE_ACCOUNT").get(),
    user=Secret("SNOWFLAKE_USER").get(),
    password=Secret("SNOWFLAKE_PASSWORD").get(),
    ...
)
with Flow("weather_notifications", schedule=schedule) as flow:
    snowflake_query.map(...)
k

Kevin Kho

05/10/2022, 8:28 PM
You should be able to map it yes. But you need to handle concurrent connections to Snowflake, otherwise you may get errors
j

Jason

05/10/2022, 8:46 PM
What gets mapped, in that case? Am I setting up a callable like:
Copy code
@task
def snowflake_query(name, s3_path):
    return SnowflakeQuery(...)
k

Kevin Kho

05/10/2022, 8:48 PM
No in Andrew’s example, it would be:
Copy code
with Flow():
    snowflake_query.map(query=list_of_queries)
For example
And then the stuff in the init is preserved across the mapping
j

Jason

05/10/2022, 8:49 PM
Ah, the kwargs. I see. OK interesting
4 Views