https://prefect.io
Join Slack
I have a flow that executes sql query on redshift using prefect-sqlalchemy and my queries are always...
# ask-community
r
I have a flow that executes sql query on redshift using prefect-sqlalchemy and my queries are always aborted
✅ 1
j
It might be helpful to set the default logging level to debug instead of info. “to change the default logging levels for Prefect to 
DEBUG
, you can set the environment variable 
PREFECT_LOGGING_LEVEL="DEBUG"
.” https://docs.prefect.io/concepts/logs/
r
HI Jeff, even with the DEBUG logging level, I see no error
when I run python flow_script
k
moving to thread: Rohith [3:09 AM] how can I find the causes of the abort
@Rohith just to clarify, are you able to run your query directly against the database you're querying. Does that fail as well or is that successful?
r
the query has been successful,
I have used redshift+redshift_connector (by setting this value in sqlalchemy block) to connect to redshift and execute the queries
commit was not being called after sqlalchemy_execute
I had to use a connection like sqlaldb_block.get_engine()
dbengine = database_block.get_engine()
with dbengine.begin() as conn:
r = conn.execute(f"select count(*) from kochava.kochava_{cprefix}").first()[0]
print(f"Record count for kochava_{cprefix} before insert: {r}")
conn.execute(
f"CREATE TABLE IF NOT EXISTS kochava.temp_{cprefix} (LIKE kochava.kochava_{cprefix})"
5 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by