Hey gang - I run a data engineering bootcamp and we are looking for a couple more externship partners for our students. What the org gets: • 20 hrs of work per week (from each 2-intern team) • 3 months duration (with the option to extend upon mutual agreement) • Interns are college grads with analytical backgrounds Company requirements: • Define a clear and meaningful project • Provide technical guidance on tech stack and engineering practices Reach out if it might be a fit!

Hey folks, this article is a reflection on what I learned in a recent conversation with Ole Olesen-Bagneux about his upcoming book “The Enterprise Data Catalog” with O'Reilly Media. Also I am writing this because I want to share what I learned with you and dive deeper into how to build a #datacatalog at scale. If you want to dig deeper check out episode 038 of the Discovering Data podcast! Links inside👇 #discoveringdata #datamanagement #podcast https://www.linkedin.com/pulse/how-we-organize-data-defines-can-search-loris-marini

Happy Meme Friday!

Python is the best!

Happy Friday!

Congrats to the team on all their hard work!

Happy Meme-Friday! 🤓

Hello, is anybody know good resources about python code convention (const, function names) I would like to follow a good python convention while coding. The same as airbnb rules for ecmascript for example. Maybe https://google.github.io/styleguide/pyguide.html should be enough ?

Happy Meme Friday! marvin duck

👋🏻 Hey folks, Going Live in 3 hours 🍿*Tune in:*

So I was reflecting on my conversation with Bill Schmarzo particularly on why most practitioners don’t get invited to business meetings. My top two lessons are that 1. we need to change our language and we MUST find a revenue connection with everything we do and 2. if we jump to implementation we don’t give people the space to share the context of what they are trying to do, and our data product will be useless. Among many things we talked about: 1. What it means to be a data leader 2. How to speak business language 3. How to find revenue connections with what we do 4. How to think like a designer to create great products 5. The missed opportunities when we jump to implementation 6. How to get people comfortable sharing what’s not working 7. The two sides of design thinking 8. The immense business benefits of capturing, maintaining, and sharing knowledge and more 9. I just released this on my website, it’s episode 039 of the Discovering Data podcast 👇 https://open.spotify.com/episode/2Eusqst7gSQ7NCfjObZDCS?si=0f2511d136844bc1

Happy Meme Friday! marvin duck

RIP to Atom you were a real one

Happy meme Friday! marvin duck

Hello everyone, I have a question about how to handle failures in flow runs: I have a basic ETL pipeline set up as a flow that is scheduled to execute daily, and it ingests data from about a 100 different endpoints. It is expected that the ingest for most of those endpoints will be successful, but also that there will always be several that are unsuccessful. But since some are unsuccessful, Prefect shows the entire flow run as a failure. Is there a way to change that? I'd prefer that it only show as a failure if none of the ingests are successful, because it's set up to alert people who use this about failures. The Flow object in Python is set up sort of like this

source_ids = ["source_1", "source_2", "source_3"]
with Flow("my_flow") as flow:
    for source_id in source_ids:
        data = extract(source_id)
        data = transform(data)
        load(data)
    return flow

Thanks in advance!

In honor of meme Friday; depending on time zone still plenty of time to push to prod! https://twitter.com/reactive_dude/status/1559904011079270404?s=20&t=S2YStzPyKO2IfndTWF9B3Q

The trillion dollar question: how do we turn data into knowledge at scale? Clean, structured, and trusted data is the goal of most data engineering teams. But data per se isn’t the end goal. We need to create a stable knowledge base for the organization. This is part 1 from my conversation with Jessica Talisman in episode 040 of the Discovering Data podcast👇 https://open.spotify.com/episode/1L2qQcDpSTAHdZzVhhXCi6?si=9c1f1eab2200450d

Happy meme Friday!

The greatest name we have had on one of our flows, Teenage Mutant....