Episode Details
Back to Episodes
AI Is Changing Data Engineering Fast
Description
In this episode of the Data Engineering Central Podcast, I sit down with Andreas Kretz to break down what is really happening in the industry right now. We go far beyond surface-level AI hype and talk about how data engineering actually works in the real world, what skills still matter, and where most engineers are wasting time.
Andreas shares his full journey from industrial IoT and working at Bosch to building one of the largest data engineering education platforms in the world, training over 2,000 students and reaching more than 100,000 engineers globally. We get into what production data systems actually look like, why most learning paths are broken, and how AI is reshaping the role of the modern data engineer.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
* We also dig into the uncomfortable truths. AI can write code, but it cannot replace thinking. Most engineers focus too much on tools and not enough on problem-solving, system design, and communication. That gap is only getting bigger.
If you are trying to figure out how to stay relevant in data engineering, or you are just getting started and want to avoid years of wasted effort, this conversation will change how you think about your career.
Today’s podcast is sponsored by Estuary.
Without them, content like this isn’t possible. The best way to support this Newsletter is to check out what Estuary has to offer and click the links below.
Build millisecond-latency, scalable, future-proof data pipelines in minutes.
Estuary is the Right-Time Data Platform that integrates all of the systems you use to produce, process, and consume data. Also, providing best-in-class CDC (Change Data Capture).
Estuary unifies today’s batch and streaming paradigms so that your systems, current and future, are synchronized around the same datasets, updating in milliseconds.
What we cover:
* Why most data engineers are learning the wrong things
* The shift from coding to problem-solving and system design
* How AI is actually changing data engineering workflows
* Why courses and tutorials are becoming less effective
* The difference between real production systems and “toy projects.”
* The future of data engineering jobs and whether AI will replace them
* Why fundamentals still matter more than ever
One of the biggest takeaways is simple. The tools will keep changing, but the problems stay the same. The engineers who win are those who understand systems, ask better questions, and connect business problems to real solutions.
Links:
* Learn Data Engineering Academy:
https://learndataengineering.com
* Sponsor: https://estuary.dev
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.