Data Modeling for Apache Kafka – Streams, Topics & More with Dani Traphagen
Episode description
Helping users be successful when it comes to using Apache Kafka® is a large part of Dani Traphagen’s role as a senior systems engineer at Confluent. Whether she’s advising companies on implementing parts of Kafka or rebuilding their systems entirely from the ground up, Dani is passionate about event-driven architecture and the way streaming data provides real-time insights on business activity.
She explains the concept of a stream, topic, key, and stream-table duality, and how each of these pieces relate to one another. When it comes to data modeling, Dani covers importance business requirements, including the need for a domain model, practicing domain-driven design principles, and bounded context. She also discusses the attributes of data modeling: time, source, key, header, metadata, and payload, in addition to exploring the significance of data governance and lineage and performing joins.
EPISODE LINKS
- Convert from table to stream and stream to table
- Distributed, Real-Time Joins and Aggregations on User Activity Events Using Kafka Streams
- KSQL in Action: Real-Time Streaming ETL from Oracle Transactional Data
- KSQL in Action: Enriching CSV Events with Data from RDBMS into AWS
- Journey to Event Driven – Part 4: Four Pillars of Event Streaming Microservices
- Join the Confluent Community Slack
- Fully managed Apache Kafka as a service! Try free.
SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites
Artwork by Phil Vo
- 🎧 Subscribe to Confluent Developer wherever you listen to podcasts.
- ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
- 👍 If you enjoyed this, please leave us a rating.
- 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.