Nick Dearden explains the five stages of streaming maturity. They are not denial, anger, bargaining, depression and acceptance—that’s the Kübler-Ross model, and it’s for bad things. This one is for awesome things, and takes you from the first streaming project you ever build all the way to a state where an entire organization is transformed to think in terms of real-time, event-driven systems. If you have ever found yourself trying to get streaming technology adopted, this episode is for you! EP...
Mar 18, 2019•37 min•Ep 24•Transcript available on Metacast Kubernetes provides all the building blocks needed to run stateful workloads, but creating a truly enterprise-grade Apache Kafka® platform that can be used in production is not always intuitive. In this episode, Tim Berglund and Viktor Gamov address some of the challenges and pitfalls of managing Kafka on Kubernetes at scale. They also share lessons learned from the development of the Confluent Operator for Kubernetes, and answer questions like: -What is Kubernetes? -What are stateful workloads?...
Mar 11, 2019•42 min•Ep 23•Transcript available on Metacast We all know that feeling of waiting when your ride is running late. Leslie Kurt shares about how you can use KSQL to calculate the difference between the expected arrival time and real-time updates of a bus as it executes its route. Listen as Leslie walks you through fundamental concepts like KTables, Kafka Streams, persistent queries and Confluent MQTT Proxy, as well as other use cases that involve a similar mechanism of capturing Unix timestamps and performing a stream processing operation on ...
Mar 04, 2019•19 min•Ep 22•Transcript available on Metacast When you are dealing with streaming data, it might seem like tables are things that dwell in the far-off land of relational databases, outside of Apache Kafka and your event streaming system. But then the Kafka Streams API gives us the KTable abstraction, which lets us create tabular views of data in Kafka topics. Apache Kafka 2.1 featured an interesting change to the table API—commonly known to the world as KIP-328 —that gives you better control over how updates to tables are emitted into desti...
Feb 27, 2019•46 min•Ep 21•Transcript available on Metacast Tim Berglund chats with System Engineer Pascal Vantrepote about a KSQL recipe he created based on a real-life customer use case in the financial services industry. They also discuss the advantages of KSQL, such as its expressiveness and ease of deployment in places where you’re not already writing a Java application. EPISODE LINKS About KSQL Stream Processing Cookbook KSQL Recipe: Data Routing Joined with a KTable For more, you can check out ksqlDB , the successor to KSQL....
Feb 25, 2019•21 min•Ep 20•Transcript available on Metacast Want to know how Kafka Connect distributes tasks to workers? Always thought Connect rebalances could be improved? In this episode of Ask Confluent, Gwen Shapira speaks with Konstantine Karantasis, software engineer at Confluent, about the latest improvements to Kafka Connect and how to run the Confluent CLI on Windows. EPISODE LINKS Improved rebalancing for Kafka Connect Improved rebalancing for Kafka Streams The "what would Kafka do?" scenario from Mark Papadakis The future of retail ...
Feb 20, 2019•21 min•Ep 19•Transcript available on Metacast Neil Avery explores the intersection between FaaS and event streaming applications before taking a quick detour back in time to understand how we've gotten to this point in event-driven applications. He'll explain the pros and cons of FaaS, and cover how in its current state cold starts and latency concerns need to be part of the bigger picture when building streaming applications. Finally, Neil shares five rules that will help you understand how FaaS fits with the event streaming appl...
Feb 14, 2019•41 min•Ep 18•Transcript available on Metacast Tim Berglund hosts Developer Advocate Ricardo Ferreira to discuss the concept of infrastructure as code, as well as the differences between Terraform, Ansible, Puppet and Chef. They also chat about why Terraform is such a big deal, some of the challenges involved with learning it and how Confluent leverages Terraform to achieve multi-cloud support for Confluent Cloud and tools for Confluent Platform. EPISODE LINKS Terraform Tools for Confluent Cloud Clusters Fully managed Apache Kafka as a servi...
Jan 23, 2019•29 min•Ep 17•Transcript available on Metacast Gwen asks: What happens when garbage collection causes Kafka to pause? And how do we run a Schema Registry cluster? We’ll find out in this episode of Ask Confluent. In "Ask Confluent," Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS ZooKeeper connection timeout configuration: zookeeper.connection.timeout.ms, as we said, this defaults to 6,000 Schema Registry failover instructions Wa...
Jan 08, 2019•15 min•Ep 16•Transcript available on Metacast Gwen is joined in studio by special guest Guozhang Wang, Kafka Streams pioneer and engineering lead at Confluent. He’ll talk to us about standby tasks and how one deserializes message headers. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Documentation of standby tasks, including configs Events with different schema in same topic How to populate a database from Ka...
Dec 18, 2018•22 min•Ep 15•Transcript available on Metacast Gwen is joined in studio by special guest Matthias J. Sax, a software engineer at Confluent. He’ll talk to us about Kafka consumers and Kafka Streams failover. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast
Dec 03, 2018•24 min•Ep 14•Transcript available on Metacast Gwen is joined in studio by special guest Jason Gustafson, a Kafka PMC member and engineer at Confluent. He’ll talk to us about the big questions on Kafka architecture— number of partitions and exactly once. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Hardening Kafka Replication Kafka open issues Watch the video version of this podcast...
Nov 05, 2018•22 min•Ep 13•Transcript available on Metacast Neha Narkhede leads a panel discussion at Kafka Summit SF 2018 with Kevin Scott (CTO, Microsoft), Julia Grace (Head of Infrastructure Engineering, Slack), Martin Kleppman (Researcher, U. of Cambridge), Jay Kreps (Co-founder and CEO, Confluent) and Neha Narkhede (Co-founder and CTO at Confluent).
Oct 18, 2018•35 min•Ep 12•Transcript available on Metacast Tim Berglund interviews Bill Bejeck about the Kafka Streams API and his new book, Kafka Streams in Action .
Sep 27, 2018•49 min•Ep 11•Transcript available on Metacast KSQL 5.0 now supports stream-stream, stream-table and table-table joins. Tim Berglund interviews Hojjat Jafarpour about all three join types, how they work, what their limitations are and the new kinds of operations they unlock. For more, you can check out ksqlDB , the successor to KSQL.
Sep 20, 2018•29 min•Ep 10•Transcript available on Metacast Gwen is joined in studio by co-host Tim Berglund and special guest, Viktor Gamov, a new member of Confluent’s Developer Experience Team specializing in Kafka, KSQL and Kubernetes. In this episode, we’ll find out: Does Viktor know what he’s talking about? EPISODE LINKS Watch the video version of this podcast
Sep 10, 2018•31 min•Ep 9•Transcript available on Metacast A discussion about how people actually use KSQL with Nick Dearden, stream processing expert at Confluent. Try KSQL! For more, you can check out ksqlDB , the successor to KSQL.
Sep 06, 2018•32 min•Ep 8•Transcript available on Metacast Interesting data isn't a polite little list of scalar types. Sometimes you have more complex structures and things like nesting. We'll see how KSQL supports that today as Tim Berglund discusses nested data in KSQL with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. EPISODE LINKS KSQL demos and info KSQL GitHub KSQL Slack (#ksql channel) For more, you can check out ksqlDB , the successor to KSQL....
Aug 29, 2018•13 min•Ep 7•Transcript available on Metacast KSQL has a solid library of built-in functions, but no library is ever good enough. What if you want to write your own? We’ll learn how today with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. For more, you can check out ksqlDB , the successor to KSQL.
Aug 24, 2018•19 min•Ep 6•Transcript available on Metacast Want to see a feature implemented in KSQL or other Kafka-related project? Gwen answers your questions from YouTube and walks through how to use GitHub issues to request features. This is the episode #4 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast
Aug 16, 2018•14 min•Ep 5•Transcript available on Metacast Ever wonder what actually goes on when you run a KSQL query? Today, we take a deep dive into KSQL with Hojjat Jafarpour, a software engineer on the KSQL team at Confluent. For more, you can check out ksqlDB , the successor to KSQL.
Aug 13, 2018•33 min•Ep 4•Transcript available on Metacast Tim Berglund and Gwen Shapira have a discussion with Koelli Mungee (Customer Operations Lead, Confluent) and cover the latest Apache Kafka upgrades, cloud APIs, and data durability. This is episode #3 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube, and elsewhere. EPISODE LINKS Watch the video version of this podcast Fully managed Apache Kafka as a service! Try free....
Jul 20, 2018•23 min•Ep 3•Transcript available on Metacast Gwen Shapira answers your questions and interviews Sam Hecht (Head of Support, Confluent). This is the second episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast
Jul 02, 2018•24 min•Ep 2•Transcript available on Metacast Tim Berglund and Gwen Shapira discuss Kubernetes, Confluent Operator, Kafka, KSQL, and more. This is the first episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. EPISODE LINKS Watch the video version of this podcast
Jun 20, 2018•23 min•Ep 1•Transcript available on Metacast 
