Has anyone done anything with clickhouse and prefect that minds sharing some of their tasks how they are inserting data? Current process is we have a spark job that does our ETL and inserts into our sql server database. but I'm currently exploring clickhouse as our new data store and looking at prefect to kinda orchestrate and monitor our ETLs

Heh, I moved from Spark to Clickhouse too a while back Which client are you using? I'm using aiochclient, then just run SQL as usual under ChClient.execute(), seems to work well

Yes same for us. Looking to move to prefect 2 in the upcoming month

I just started reviewing the clickhouse python interfaces. The API I'm writing is in c# and im using the Clickhouse.Client package. but it's been awhile since ive written python outside of simple scripts so just trying to get some ideas