Todd de Quincey
05/24/2022, 9:45 AMTodd de Quincey
05/24/2022, 9:45 AMresult = snowflake_query(
query="put file:///data.csv @my_csv_stage",
snowflake_credentials=SNOWFLAKE_CREDENTIALS,
)
I get the following error
11:40:45.794 | ERROR | Task run 'snowflake_query-eb69c8ef-0' - Encountered exception during execution:
Traceback (most recent call last):
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/prefect/engine.py", line 796, in orchestrate_task_run
result = await task.fn(*args, **kwargs)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/prefect_snowflake/database.py", line 68, in snowflake_query
response = cursor.execute_async(query, params=params)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 804, in execute_async
return self.execute(*args, **kwargs)
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 755, in execute
sf_file_transfer_agent.execute()
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 355, in execute
self._parse_command()
File "/Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 852, in _parse_command
response = self._ret["data"]
KeyError: 'command'
Setting a breakpoint at /Users/todddequincey/repos/midnite/data/prefect/.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py::854
and inspecting the response = _self_._ret["data"]
object, there is indeed no command
object in there.
Not sure if this is a user error, but haven’t been able to find much in the way of docs or examples for this.
Note: I have manually checked in snowflake that the user and role has appropriate access to the stage. And inspecting the os path, the file is accessible from the code.Anna Geller
Todd de Quincey
05/24/2022, 10:57 AMfrom prefect_snowflake.database import snowflake_query
from prefect import flow
from util.const import AWS_CREDENTIALS, SNOWFLAKE_CREDENTIALS, S3_RAW_BUCKET
@flow
def snowflake_pipeline():
result = snowflake_query(
query="put file:///data.csv @my_csv_stage",
snowflake_credentials=SNOWFLAKE_CREDENTIALS,
)
return None
state = snowflake_pipeline()
Todd de Quincey
05/24/2022, 10:57 AMTodd de Quincey
05/24/2022, 10:58 AMAnna Geller
Anna Geller
Todd de Quincey
05/24/2022, 11:23 AMAnna Geller
Todd de Quincey
05/24/2022, 11:24 AMTodd de Quincey
05/24/2022, 11:24 AMAnna Geller
Anna Geller
Anna Geller
Anna Geller
Todd de Quincey
05/24/2022, 11:28 AMTodd de Quincey
05/24/2022, 11:30 AMAnna Geller
Anna Geller