Todd de Quincey
05/24/2022, 9:45 AMresult = snowflake_query(
query="put file:///data.csv @my_csv_stage",
snowflake_credentials=SNOWFLAKE_CREDENTIALS,
)
I get the following error
11:40:45.794 | ERROR | Task run 'snowflake_query-eb69c8ef-0' - Encountered exception during execution:
Traceback (most recent call last):
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/prefect/engine.py", line 796, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/prefect_snowflake/database.py", line 68, in snowflake_query
response = cursor.execute_async(query, params=params)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 804, in execute_async
return self.execute(*args, **kwargs)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 755, in execute
sf_file_transfer_agent.execute()
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 355, in execute
self._parse_command()
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 852, in _parse_command
response = self._ret["data"]
KeyError: 'command'
Setting a breakpoint at /Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py::854
and inspecting the response = _self_._ret["data"]
object, there is indeed no command
object in there.
Not sure if this is a user error, but haven’t been able to find much in the way of docs or examples for this.
Note: I have manually checked in snowflake that the user and role has appropriate access to the stage. And inspecting the os path, the file is accessible from the code.Anna Geller
05/24/2022, 10:54 AMTodd de Quincey
05/24/2022, 10:57 AMfrom prefect_snowflake.database import snowflake_query
from prefect import flow
from util.const import AWS_CREDENTIALS, SNOWFLAKE_CREDENTIALS, S3_RAW_BUCKET
@flow
def snowflake_pipeline():
result = snowflake_query(
query="put file:///data.csv @my_csv_stage",
snowflake_credentials=SNOWFLAKE_CREDENTIALS,
)
return None
state = snowflake_pipeline()
Anna Geller
05/24/2022, 11:20 AMTodd de Quincey
05/24/2022, 11:23 AMAnna Geller
05/24/2022, 11:23 AMTodd de Quincey
05/24/2022, 11:24 AMAnna Geller
05/24/2022, 11:24 AMTodd de Quincey
05/24/2022, 11:28 AMAnna Geller
05/24/2022, 11:32 AM