https://prefect.io logo
Title
r

Rohith

11/08/2022, 11:05 AM
I have a flow that executes sql query on redshift using prefect-sqlalchemy and my queries are always aborted
1
j

Jeff Hale

11/08/2022, 7:00 PM
It might be helpful to set the default logging level to debug instead of info. “to change the default logging levels for Prefect to
DEBUG
, you can set the environment variable
PREFECT_LOGGING_LEVEL="DEBUG"
.” https://docs.prefect.io/concepts/logs/
r

Rohith

11/09/2022, 6:03 AM
HI Jeff, even with the DEBUG logging level, I see no error
when I run python flow_script
k

Kalise Richmond

11/09/2022, 5:25 PM
moving to thread: Rohith [3:09 AM] how can I find the causes of the abort
@Rohith just to clarify, are you able to run your query directly against the database you're querying. Does that fail as well or is that successful?
r

Rohith

11/10/2022, 7:57 AM
the query has been successful,
I have used redshift+redshift_connector (by setting this value in sqlalchemy block) to connect to redshift and execute the queries
commit was not being called after sqlalchemy_execute
I had to use a connection like sqlaldb_block.get_engine()
dbengine = database_block.get_engine()
with dbengine.begin() as conn:
r = conn.execute(f"select count(*) from kochava.kochava_{cprefix}").first()[0]
print(f"Record count for kochava_{cprefix} before insert: {r}")
conn.execute(
f"CREATE TABLE IF NOT EXISTS kochava.temp_{cprefix} (LIKE kochava.kochava_{cprefix})"