Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

video.gif

So this gif above was part of a larger project (which i finally finished) that was the final task of a course I took in data engineering.

I basically pulled as much data as I could from r/dataengineering subreddit, built pipelines with prefect and dbt, did some NLP shenanigans and displayed the results in a Superset dashboard.

You can find the link to the <https://github.com/wtfzambo/subreddit-analytics|project here> and the <http://34.110.171.173/superset/dashboard/1/?preselect_filters=%7B%7D&amp;standalone=true&amp;native_filters_key=vg_Fem5WRlplqM-jS4x75grLULDwQhqUnCL4ix3HPft6VMQ-okiiiHtnhWDRQeXq|dashboard here>.
Any feedback is very welcome!

dang this is awesome!! will check out your NLP shenanigans :eyes:

Thanks for the feedback, glad you enjoyed it !

(tbf the NLP shenanigans are nothing more than playing around with TFIDF and some custom logic i came up with on the fly :sweat_smile:)