Kubernetes provides all the building blocks needed to run stateful workloads, but creating a truly enterprise-grade Apache Kafka® platform that can be used in production is not always intuitive. In this episode, Tim Berglund and Viktor Gamov address some of the challenges and pitfalls of managing Kafka on Kubernetes at scale. They also share lessons learned from the development of the Confluent Operator for Kubernetes, and answer questions like: -What is Kubernetes? -What are stateful workloads?...
Mar 11, 2019•42 min•Season 1Ep. 23
We all know that feeling of waiting when your ride is running late. Leslie Kurt shares about how you can use KSQL to calculate the difference between the expected arrival time and real-time updates of a bus as it executes its route. Listen as Leslie walks you through fundamental concepts like KTables, Kafka Streams, persistent queries and Confluent MQTT Proxy, as well as other use cases that involve a similar mechanism of capturing Unix timestamps and performing a stream processing operation on ...
Mar 04, 2019•19 min•Season 1Ep. 22
When you are dealing with streaming data, it might seem like tables are things that dwell in the far-off land of relational databases, outside of Apache Kafka and your event streaming system. But then the Kafka Streams API gives us the KTable abstraction, which lets us create tabular views of data in Kafka topics. Apache Kafka 2.1 featured an interesting change to the table API—commonly known to the world as KIP-328 —that gives you better control over how updates to tables are emitted into desti...
Feb 27, 2019•46 min•Season 1Ep. 21
Tim Berglund chats with System Engineer Pascal Vantrepote about a KSQL recipe he created based on a real-life customer use case in the financial services industry. They also discuss the advantages of KSQL, such as its expressiveness and ease of deployment in places where you’re not already writing a Java application. EPISODE LINKS About KSQL Stream Processing Cookbook KSQL Recipe: Data Routing Joined with a KTable For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim...
Feb 25, 2019•21 min•Season 1Ep. 20
Want to know how Kafka Connect distributes tasks to workers? Always thought Connect rebalances could be improved? In this episode of Ask Confluent, Gwen Shapira speaks with Konstantine Karantasis, software engineer at Confluent, about the latest improvements to Kafka Connect and how to run the Confluent CLI on Windows. EPISODE LINKS Improved rebalancing for Kafka Connect Improved rebalancing for Kafka Streams The "what would Kafka do?" scenario from Mark Papadakis The future of retail at Nordstr...
Feb 20, 2019•21 min•Season 1Ep. 19
Neil Avery explores the intersection between FaaS and event streaming applications before taking a quick detour back in time to understand how we've gotten to this point in event-driven applications. He'll explain the pros and cons of FaaS, and cover how in its current state cold starts and latency concerns need to be part of the bigger picture when building streaming applications. Finally, Neil shares five rules that will help you understand how FaaS fits with the event streaming application. E...
Feb 14, 2019•41 min•Season 1Ep. 18
Tim Berglund hosts Developer Advocate Ricardo Ferreira to discuss the concept of infrastructure as code, as well as the differences between Terraform, Ansible, Puppet and Chef. They also chat about why Terraform is such a big deal, some of the challenges involved with learning it and how Confluent leverages Terraform to achieve multi-cloud support for Confluent Cloud and tools for Confluent Platform. EPISODE LINKS Terraform Tools for Confluent Cloud Clusters Fully managed Apache Kafka as a servi...
Jan 23, 2019•29 min•Season 1Ep. 17
Gwen asks: What happens when garbage collection causes Kafka to pause? And how do we run a Schema Registry cluster? We’ll find out in this episode of Ask Confluent. In "Ask Confluent," Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS ZooKeeper connection timeout configuration: zookeeper.connection.timeout.ms, as we said, this defaults to 6,000 Schema Registry failover instructions Watch the vi...
Jan 08, 2019•15 min•Season 1Ep. 16
Gwen is joined in studio by special guest Guozhang Wang, Kafka Streams pioneer and engineering lead at Confluent. He’ll talk to us about standby tasks and how one deserializes message headers. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Documentation of standby tasks, including configs Events with different schema in same topic How to populate a database from Kafka and so...
Dec 18, 2018•22 min•Season 1Ep. 15
Gwen is joined in studio by special guest Matthias J. Sax, a software engineer at Confluent. He’ll talk to us about Kafka consumers and Kafka Streams failover. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed...
Dec 03, 2018•24 min•Season 1Ep. 14
Gwen is joined in studio by special guest Jason Gustafson, a Kafka PMC member and engineer at Confluent. He’ll talk to us about the big questions on Kafka architecture— number of partitions and exactly once. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Hardening Kafka Replication Kafka open issues Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi...
Nov 05, 2018•22 min•Season 1Ep. 13
Neha Narkhede leads a panel discussion at Kafka Summit SF 2018 with Kevin Scott (CTO, Microsoft), Julia Grace (Head of Infrastructure Engineering, Slack), Martin Kleppman (Researcher, U. of Cambridge), Jay Kreps (Co-founder and CEO, Confluent) and Neha Narkhede (Co-founder and CTO at Confluent). SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Deve...
Oct 18, 2018•35 min•Season 1Ep. 12
Tim Berglund interviews Bill Bejeck about the Kafka Streams API and his new book, Kafka Streams in Action . SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Developer wherever you listen to podcasts. ▶️ Subscribe on YouTube , and hit the 🔔 to catch new episodes. 👍 If you enjoyed this, please leave us a rating. 🎧 Confluent also has a podcast for ...
Sep 27, 2018•49 min•Season 1Ep. 11
KSQL 5.0 now supports stream-stream, stream-table and table-table joins. Tim Berglund interviews Hojjat Jafarpour about all three join types, how they work, what their limitations are and the new kinds of operations they unlock. For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Developer wh...
Sep 20, 2018•29 min•Season 1Ep. 10
Gwen is joined in studio by co-host Tim Berglund and special guest, Viktor Gamov, a new member of Confluent’s Developer Experience Team specializing in Kafka, KSQL and Kubernetes. In this episode, we’ll find out: Does Viktor know what he’s talking about? EPISODE LINKS Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Co...
Sep 10, 2018•31 min•Season 1Ep. 9
A discussion about how people actually use KSQL with Nick Dearden, stream processing expert at Confluent. Try KSQL! For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Developer wherever you listen to podcasts. ▶️ Subscribe on YouTube , and hit the 🔔 to catch new episodes. 👍 If you enjoyed ...
Sep 06, 2018•32 min•Season 1Ep. 8
Interesting data isn't a polite little list of scalar types. Sometimes you have more complex structures and things like nesting. We'll see how KSQL supports that today as Tim Berglund discusses nested data in KSQL with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. EPISODE LINKS KSQL demos and info KSQL GitHub KSQL Slack (#ksql channel) For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited...
Aug 29, 2018•13 min•Season 1Ep. 7
KSQL has a solid library of built-in functions, but no library is ever good enough. What if you want to write your own? We’ll learn how today with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Developer wherever you li...
Aug 24, 2018•19 min•Season 1Ep. 6
Want to see a feature implemented in KSQL or other Kafka-related project? Gwen answers your questions from YouTube and walks through how to use GitHub issues to request features. This is the episode #4 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagh...
Aug 16, 2018•14 min•Season 1Ep. 5
Ever wonder what actually goes on when you run a KSQL query? Today, we take a deep dive into KSQL with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. For more, you can check out ksqlDB , the successor to KSQL. SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subscribe to Confluent Developer wherever you listen to podcasts. ▶️ Subscribe on YouTube , ...
Aug 13, 2018•33 min•Season 1Ep. 4
Tim Berglund and Gwen Shapira have a discussion with Koelli Mungee (Customer Operations Lead, Confluent) and cover the latest Apache Kafka upgrades, cloud APIs, and data durability. This is episode #3 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube, and elsewhere. EPISODE LINKS Watch the video version of this podcast Fully managed Apache Kafka as a service! Try free. SEASON 2 Hosted by Tim Berglund, Adi Polak and...
Jul 20, 2018•23 min•Season 1Ep. 3
Gwen Shapira answers your questions and interviews Sam Hecht (Head of Support, Confluent). This is the second episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subsc...
Jul 02, 2018•24 min•Season 1Ep. 2
Tim Berglund and Gwen Shapira discuss Kubernetes, Confluent Operator, Kafka, KSQL, and more. This is the first episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo 🎧 Subs...
Jun 20, 2018•23 min•Season 1Ep. 1
