set the channel description: Discussion about the data ecosystem: from database to analytics, and everything in between.

I'll start, Clickhouse rocks 😁

As of today, Omni is now commercially available which is pretty cool - the release coincides with a bunch of product updates as well including new permission features: https://www.exploreomni.com/blog/user-permissions

^lol great minds

LOL omni fans unite ✊

Interested to hear if anyone is using the likes of Atlan/Transform/Amundsen for democratizing their data (internally)

duckdb anyone ? I replaced a postgres db (used for OLAP) with duckdb + parquet file

Retool is my favorite new toy. Building out workflows where users interact with the data. In our case, we have nurses doing clinical validation of our AI output and saving their validation and notes back in to the db. Real human-in-the-loop stuff, without a SE team to build the UI for us. This is the next gen of BI tools, imo.

I have been looking at Hamilton package to manage feature store as well as feature dependency. It has it's own execution engine as well. Has anyone thought or tried to use it with Prefect?

dbt Cloud has webhooks now

From the perspective of someone not working in a major US-based company, the costs for some of the tools in the ML/Data Science/Data Engineering space are kinda bonkers tbh.

This may be old news for some, but pandas 2 is built on Arrow. This might be the thing that creates a tipping point for Arrow's adoption. It feels like Wes has been encouraging it for years.

Would be interested to hear how many of you are using custom validation vs out of the box packages, as in this use case.

Any Hex.Tech/Count.Co/other users in here - hex have just released EU hosting which opens it up to me but was wondering if there is anything better out there?

Hi all, doing some research and was curious had anyone used MS SQL server with Prefect before?

Hello, (how) does prefect work with asyncio? E.g. with asyncpg?

This is a strange app to see here. ☝️

hi all, I would appricate any with an issue i'm facing, described here: https://discourse.prefect.io/t/docker-container-infrastructure-not-being-used/3458

I have a docker run infrastructure block with a docker image. I need to get the container's host name (where the agent is running) inside the flow code. any idea how to achive the above ?