Bryan
09/20/2024, 9:08 PMMarvin
09/20/2024, 9:08 PMbash
pip install prefect pandas snowflake-connector-python
Here's a sample Prefect flow to query Snowflake and return the results as a DataFrame:
python
import pandas as pd
from prefect import flow, task
from prefect_snowflake.database import SnowflakeConnector
from snowflake.connector.pandas_tools import write_pandas
@task
def fetch_data_as_dataframe(snowflake_conn_info: dict, query: str) -> pd.DataFrame:
with SnowflakeConnector(**snowflake_conn_info) as connector:
df = connector.fetch_as_dataframe(query)
return df
@flow
def snowflake_to_dataframe_flow():
snowflake_conn_info = {
"user": "YOUR_USER",
"password": "YOUR_PASSWORD",
"account": "YOUR_ACCOUNT",
"warehouse": "YOUR_WAREHOUSE",
"database": "YOUR_DATABASE",
"schema": "YOUR_SCHEMA"
}
query = "SELECT * FROM your_table"
df = fetch_data_as_dataframe(snowflake_conn_info, query)
print(df)
if __name__ == "__main__":
snowflake_to_dataframe_flow()
Replace the placeholder values in snowflake_conn_info
with your actual Snowflake connection details and specify your query. This sample demonstrates how to fetch data from Snowflake and return it as a pandas DataFrame within a Prefect flow.
For more details, refer to the Prefect 2.x Snowflake integration documentation:
https://docs-2.prefect.io/latest/integrations/prefect-snowflake/
If you need help with anything else or face any issues, feel free to ask!