When it comes to adopting event-driven architectures, a couple of key considerations often arise: the way that an asynchronous core interacts with external synchronous systems and the question of “how do I refactor my monolith into services?” Daniel Jagielski, a consultant working as a tech lead/dev manager at VirtusLab for Tesco, recounts how these very themes emerged in his work with European clients. Through observing organizations as they pivot toward becoming real time and event driven, Dan...
Feb 22, 2021•48 min•Ep 145•Transcript available on Metacast Viktor Gamov (Developer Advocate, Confluent) joins Tim Berglund on this episode to talk all about Spring and Apache Kafka®. Viktor’s main focus lately has been helping developers build their apps with stream processing, and helping them do this effectively with different languages. Viktor recently hosted an online Spring Boot workshop that turned out to be a lot of fun. This means it was time to get him back on the show to talk about this all-important framework and how it integrates with Kafka ...
Feb 17, 2021•45 min•Ep 144•Transcript available on Metacast Confluent Platform 6.1 further simplifies management tasks for Apache Kafka® operators. Based on Apache Kafka 2.7, this release provides even higher availability for enterprises who are using Kafka as the central backbone for their business-critical applications. Confluent Platform 6.1 delivers enhancements that reduce the risk of downtime, simplify operations and streamline the user experience, as well as improve visibility and control with centralized management. EPISODE LINKS Check out the re...
Feb 10, 2021•10 min•Ep 143•Transcript available on Metacast Nationwide Building Society, a financial institution in the United Kingdom with 137 years of history and over 18,000 employees, relies on Apache Kafka® for their event streaming needs. But how did this come to be? In this episode, Tim Berglund talks with Rob Jackson (Principal Architect, Nationwide) about their Kafka adoption journey as they celebrate two years in production. Nationwide chose to adopt Kafka as a central part of their information architecture in order to integrate microservices. ...
Feb 08, 2021•49 min•Ep 142•Transcript available on Metacast Coming up with an honest test built on open source tools in an easily documented, replicable environment for a distributed system like Apache Kafka® is not simple. Alok Nikhil (Cloud Native Engineer, Confluent) shares about getting Kafka in the cloud and how best to leverage Confluent Cloud for high performance and scalability. His blog post “Benchmarking Apache Kafka, Apache Pulsar, and RabbitMQ: Which is the Fastest?” discusses how Confluent tested Kafka’s performance on the latest cloud hardw...
Feb 01, 2021•51 min•Ep 141•Transcript available on Metacast Tim Berglund picks the brain of a distributed systems engineer, Guozhang Wang, tech lead in the Streaming department of Confluent. Guozhang explains what compelled him to join the Stream Processing team at Confluent coming from the Apache Kafka® core infrastructure. He reveals what makes the best distributed systems infrastructure engineers tick and how to prepare to take on this kind of role—solving failure scenarios, a satisfying challenge. One challenge in distributed systems is achieving agr...
Jan 25, 2021•45 min•Ep 140•Transcript available on Metacast Confluent Cloud and Confluent Platform run efficiently largely because of the dedication of the Developer Productivity (DevProd) team, formerly known as the Tools team. Mohinish Shaikh (Software Engineer, Confluent) talks to Tim Berglund about how his team builds the software tooling and automation for the entire event streaming platform and ensures seamless delivery of several engineering processes across engineering and the rest of the org. With the right tools and the right data, developer pr...
Jan 20, 2021•34 min•Ep 139•Transcript available on Metacast What’s it like being a Microsoft Azure Cloud advocate working with Apache Kafka® and change data capture (CDC) solutions? Abhishek Gupta would know! At Microsoft, Abhishek focuses his time on Kafka, databases, Kubernetes, and open source projects. His experience in a wide variety of roles ranging from engineering, consulting, and product management for developer-focused products has positioned him well for developer advocacy, where he is now. Switching gears, Abhishek proceeds to break down the ...
Jan 11, 2021•43 min•Ep 138•Transcript available on Metacast Coming out of a whirlwind year for the event streaming world, Tim Berglund sits down with Gwen Shapira (Engineering Leader, Confluent), Ben Stopford (Senior Director, Office of the CTO, Confluent), and Michael Noll (Principal Technologist, Office of the CTO, Confluent) to take a guess at what 2021 will bring. The experts share what they believe will happen for analytics, frameworks, multi-cloud services, stream processing, and other topics important to the event streaming space. These Apache Kaf...
Jan 06, 2021•45 min•Ep 137•Transcript available on Metacast It’s one thing to know how to use Apache Kafka® and another to prove it to the world that you know. Niamh O’Byrne (Certification Manager, Confluent) and Barry Ballard (Senior Technical Trainer, Confluent) discuss Confluent’s Certification program, including sample test questions, bootcamp, exam details, Kafka training, and getting the necessary practical hands-on experience. It’s no secret that the entire world of work has changed, and now we expect to communicate across a vast number of digital...
Dec 28, 2020•44 min•Ep 136•Transcript available on Metacast How do you use Apache Kafka®, Confluent Platform, and Confluent Cloud for DevOps? Integration Architects Rick Spurgeon and Allison Walther share how, including a custom tool they’ve developed for this very purpose. First, Rick and Allison share their perspective of what it means to be a DevOps engineer. Mixing development and operations skills to deploy, manage, monitor, audit, and maintain distributed systems. DevOps is multifaceted and can be compared to glue, in which you’re stitching softwar...
Dec 22, 2020•46 min•Ep 135•Transcript available on Metacast Apache Kafka® 2.7 is here! Here are the key Kafka Improvement Proposals (KIPs) and updates in this release, presented by Tim Berglund. KIP-497 adds a new inter-broker API to alter in-sync replicas (ISRs). Every partition leader maintains the ISR list or the list of ISRs. KIP-497 is also related to the removal of ZooKeeper. KIP-599 has to do with throttling the rate of creating topics, deleting topics, and creating partitions. This KIP will add a new feature called the controller mutation rate. K...
Dec 21, 2020•11 min•Ep 134•Transcript available on Metacast Chris Richardson, creator of the original Cloud Foundry, maintainer of microservices.io and author of “Microservices Patterns,” discovered cloud computing in 2006 during an Amazon talk about APIs for provisioning servers. At this time, you could provision 20 servers and pay 10 cents per hour. This blew his mind and led him in 2008 to create the original Cloud Foundry, a PaaS for deploying Java applications on EC2. One of the original Cloud Foundry’s earliest success stories was a digital marketi...
Dec 16, 2020•48 min•Ep 133•Transcript available on Metacast We have all heard of Porsche, but did you know that Porsche utilizes event streaming with Apache Kafka®? Today, Sridhar Mamella (Platform Manager, Data Streaming Platforms, Porsche) discusses how Kafka’s event streaming technology powers Porsche through Streamzilla. With the modern Porsche car having 150–200 sensors, Sridhar dives into what Streamzilla is and how it functions with Kafka on premises and in the cloud. He reveals how the first months of event streaming in production went, Porsche’s...
Dec 07, 2020•43 min•Ep 132•Transcript available on Metacast Jason Bell (Apache Kafka® DevOps Engineer, digitalis.io, and Author of “Machine Learning: Hands-On for Developers and Technical Professionals” ) delves into his 32-year journey as a DevOps engineer and how he discovered Apache Kafka. He began his voyage in hardware technology before switching over to software development. From there, he got involved in event streaming in the early 2000s where his love for Kafka started. His first Kafka project involved monitoring Kafka clusters for flight search...
Dec 02, 2020•1 hr•Ep 131•Transcript available on Metacast Multi-tenancy has been quite the topic within the Apache Kafka® community. Anna Povzner, an engineer on the Confluent team, spends most of her time working on multi-tenancy in Kafka in Confluent Cloud. Anna kicks off the conversation with Tim Berglund (Senior Director of Developer Experience, Confluent) by explaining what multi-tenancy is, why it is worthy to be desired, and advantages over single-tenant architecture. By putting more applications and use cases on the same Kafka cluster instead o...
Nov 23, 2020•44 min•Ep 130•Transcript available on Metacast Roger Hoover, one of the first engineers to work on Confluent Cloud, joins Tim Berglund (Senior Director of Developer Experience, Confluent) to chat about the evolution of Confluent Cloud, all the stages that it’s been through, and the lessons he’s learned on the way. He talks through the days before Confluent Platform was created, and how he contributed to Apache Kafka® to run it on OpenStack (the feature used to separate advertised hostnames from the internal hostnames). The Confluent Cloud co...
Nov 18, 2020•50 min•Ep 129•Transcript available on Metacast Do you ever feel like you’re short on time? Well, good news! Confluent Software Engineer Matthias J. Sax is back to discuss how event streaming has changed the game, making time management more simple yet efficient. Matthias explains what watermarking is, the reasons behind why Kafka Streams doesn’t use them, and an alternative approach to watermarking informally called the “slack time approach.” Later, Matthias discusses how you can compare “stream time,” which is the maximum timestamp observed...
Nov 12, 2020•52 min•Ep 128•Transcript available on Metacast What's it like being a distributed systems engineer? Apurva Mehta (Engineering Leader, Confluent) explains what attracted him to Apache Kafka®, the challenges and uniqueness of distributed systems, and how to excel in this industry. He dives into the complex math behind the temporal logic of actions (TLA) and shares about his experiences working at Yahoo and Linkedin, which have prepared him to be where he is today. Apurva also shares what he looks for when hiring someone to join his team. ...
Nov 02, 2020•49 min•Ep 127•Transcript available on Metacast It’s Halloween again, which means Anna McDonald (Staff Technical Account Manager, Confluent) is back for another spooktacular episode of Streaming Audio. In this episode, Anna shares six of the most spine-chilling, hair-raising Apache Kafka® JIRAs from the past year. Her job is to help hunt down problems like these and dig up skeletons like: Early death causes epoch time travel Attack of the clones Missing snapshot file leads to madness Shrink inWriteLock time to avoid maiming cluster performanc...
Oct 28, 2020•52 min•Ep 126•Transcript available on Metacast It’s the first work-from-home episode of Ask Confluent, where Gwen Shapira (Core Kafka Engineering Leader, Confluent) virtually sits down with Apache Kafka® expert Anna McDonald (Staff Technical Account Manager, Confluent) to answer questions from Twitter. Find out Anna’s favorite Kafka Improvement Proposal (KIP), which will start to use racially neutral terms in the Kafka community and in our code base, as well as answers to the following questions: If you could pick any one KIP from the backlo...
Oct 21, 2020•34 min•Ep 125•Transcript available on Metacast Wouldn’t it be awesome if there was a language as elegant as Spring Boot is as a framework? In this episode of Streaming Audio, Tim Berglund talks with Josh Long, Spring developer advocate at VMware about Kotlin, about the productivity-focused language from our friends at JetBrains, and how it works with Spring Boot to make the experience leaner, cleaner, and easy to use. Josh shares how the Spring and Kotlin teams have worked hard to make sure that Kotlin and Spring Boot are a first-class exper...
Oct 21, 2020•51 min•Ep 124•Transcript available on Metacast Neil Buesing, an Apache Kafka® community stalwart at Object Partners, spends his days building things out of Kafka and helping others do the same. Today, he discusses the concept of a CoE (center of excellence), and how a CoE is integral to attain and sustain world-class performance, business value, and success in a business. Neil talks us through how to make a CoE successful, the importance of event streaming, how to better understand streaming technologies, and how to best utilize CoE for your...
Oct 14, 2020•46 min•Ep 123•Transcript available on Metacast Ever wonder what it's like to intern at a place like Confluent? How about working with Kafka Streams and creating your own KIP? Well, that's exactly what we discuss on today's episode with Leah Thomas. Leah Thomas, who first interned as a recruiter for Confluent, quickly realized that she was enamored with the problem solving the engineering team was doing, especially with Kafka Streams. The next time she joined Confluent's intern program, she worked on the Streams team and h...
Oct 07, 2020•46 min•Ep 122•Transcript available on Metacast The feature-rich release of Confluent Platform 6.0, based on Apache Kafka® 2.6, introduces Tiered Storage, Self-Balancing Clusters, ksqlDB 0.10, Admin REST APIs, and Cluster Linking in preview. These features enhance the platform with greater elasticity, improved cost effectiveness, infinite data retention, and global availability so that you can simplify management operations, reduce the cost of adopting Kafka, and focus on building event streaming applications. EPISODE LINKS Confluent Platform...
Oct 01, 2020•14 min•Ep 121•Transcript available on Metacast Bobby Calderwood (Founder, Evident Systems) discusses event streaming, event modeling, and event-driven architecture. He describes the emerging visual language and process, how to effectively understand and teach what events are, and some of Bobby's own use cases in the field with oNote, Evident System’s new SaaS platform for event modeling. Finally, Bobby emphasizes the power of empowering and informing the community on how best to integrate event streaming with the outside world. EPISODE ...
Sep 30, 2020•57 min•Ep 120•Transcript available on Metacast Did you know that a team of 900 developers at Wix is using Apache Kafka® to maintain 1,500 microservices? Tim Berglund sits down with Natan Silnitsky (Backend Infrastructure Engineer, Wix) to talk all about how Wix benefits from using an event streaming platform. Wix (the website that’s made for building websites) is designing a platform that gives people the freedom to create, manage, and develop their web presence exactly the way they want as they look to move from synchronous to asynchronous ...
Sep 21, 2020•49 min•Ep 119•Transcript available on Metacast This year, Confluent turns six! In honor of this milestone, we are taking a very special moment to celebrate with Gwen Shapira by highlighting the top six things everyone should know about Apache Kafka®: Clients have metrics Bug fix releases/Kafka Improvement Proposals (KIPs) Idempotent producers and how they work Kafka Connect is part of Kafka and Single Message Transforms (SMTs) are worth not missing out on Cooperative rebalancing Generating sequence numbers and how Kafka changes the way you t...
Sep 15, 2020•47 min•Ep 118•Transcript available on Metacast With the explosion of real-time data, Apache Kafka and event stream processing (ESP) have grown in proliferation, with event streaming technology becoming the de facto technology transforming businesses across numerous verticals. Gwen Shapira (Engineering Leader, Confluent), Ben Stopford (Senior Director, OCTO, Confluent), and Michael Noll (Principal Technologist, Confluent) meet up to talk all about their last five years at Confluent and the changes they’ve seen in event streaming. They discuss...
Aug 31, 2020•48 min•Ep 117•Transcript available on Metacast Jay Smith helps Google Cloud users modernize their applications with serverless eventing. This helps them focus on their code instead of managing infrastructure, as well as ultra-fast deployments and reduced server costs. On today’s show, he discusses the definition of serverless, serverless eventing, data-driven vs. event-driven architecture, sources and sinks, and hybrid cloud with on-prem components. Finally, Jay shares how he sees application architecture changing in the future and where Apa...
Aug 26, 2020•47 min•Ep 116•Transcript available on Metacast 
