https://prefect.io logo
#prefect-contributors
Title
# prefect-contributors
m

Marcin Grzybowski

05/30/2022, 4:52 PM
Hi - I need a possibility to run multiple snowflake queries in one session and sometimes in a transaction I can probably easily add this - add parameter in SnowflakeQuery class: • transaction: bool = false • change query to queries that takes list of strings with queries - but this is not good for backward compatibility... I can maybe change that query is str or List[str]. Or maybe add another parameter: queries that will be taken into account when query is None.. or something But please let me know if this makes sense for you and if it has a chance to me merged when finished
a

Anna Geller

05/30/2022, 5:31 PM
Good question - to avoid breaking things, how about a separate task for that purpose? Running multiple queries in a single transaction seems like a very different use case than the current tasks and might as well be designed as a separate task.
m

Marcin Grzybowski

05/30/2022, 5:32 PM
hmm, I have just found that there is SnowflakeQueriesFromFile that executes multiple queries... but I have to check if it supports transactions, maybe it will be better to change this class
a

Anna Geller

05/30/2022, 5:33 PM
Based on your introduction, it looks like you are just getting started with Prefect - perhaps using Prefect 2.0 makes more sense for your PoC? given that we already have a snowflake collection for Prefect 2.0, you could contribute that directly there
m

Marcin Grzybowski

05/30/2022, 5:38 PM
yeah, I will ask my manager tomorrow if we can switch with our PoC to 2.0 it's not advanced so it should be quite fast I will have to check a few things before, if for example prefect2.0 flow can be run as standalone python file (we will have in a few places a need to put that kind of script in a docker container and orchestrate container without the rest of infrastructure )
a

Anna Geller

05/30/2022, 5:41 PM
there is a DockerFlowRunner and KubernetesFlowRunner for that purpose - this page references various other resources that may help
m

Marcin Grzybowski

05/30/2022, 5:43 PM
thank you
And where can I find snowflake collection? I can't see it here? https://orion-docs.prefect.io/collections/catalog/
a

Anna Geller

05/30/2022, 6:56 PM
it's here https://github.com/PrefectHQ/prefect-snowflake cc @Andrew Huang can we add Snowflake to the Collections docs/list already?
m

Marcin Grzybowski

05/30/2022, 7:08 PM
thank you
8 Views