https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Multiple clusters exist in most Kubernetes environments today, and number of clusters will increase overtime. The reasons for having multiple Kubernetes clusters are many, for example, overcoming scale limits, reducing complexity, geo separation, redundancy and having separate production, staging, and development environments. Once you have multiple K8S clusters, it can be useful to ha...
May 28, 2022•8 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Some data-intensive workloads are easier to run in Kubernetes than others. Why? What needs to improve? Join us as we deep dive with Adobe and the CNCF about how easy (or not) it is to run different types of data workloads on Kubernetes – and what is being done both inside and outside of Kubernetes to make data workloads easier. Xing Yang is a Tech Lead in the Cloud Native Storage team ...
May 28, 2022•17 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) We know from the first Data on Kubernetes Report that 90% of respondents believe Kubernetes is ready for stateful workloads, but significant challenges remain. The DoK Community continues to grow and build a unique space where people share knowledge and have conversations that are shaping the next decade of data on Kubernetes. Melissa Logan and Sylvain Kalache will discuss the growth o...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The state of the CNCF Storage options has exploded in the past few years, but if you had to choose a project to use today, how would you go about comparing each offering and choosing who to partner with for your future growth? Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within m...
May 27, 2022•9 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The Rap God project acts as a great entry point to many incoming open-source enthusiasts who are interested in learning about the cloud native ecosystem. The Rap-God project uses Kubernetes orchestration for a stateful case which is an emerging topic, the Rap God project acts as a demonstration of how to use such features of Kubernetes. The project will be using Stateful sets that’d de...
May 27, 2022•9 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) We will walk through how to build serverless event streaming applications as functions running in a function mesh on kubernetes with cloud native messaging via Apache Pulsar. In this talk, you will deploy ML functions to transform real-time data on Kubernets. Tim Spann is a Developer Advocate @ StreamNative where he works with Apache Pulsar, Apache Flink, Apache NiFi, Apache MXNet, Ten...
May 27, 2022•9 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Adobe runs dozens of Kafka clusters spread across both public (AWS and Azure) and private clouds to power the Adobe Experience Platform message bus. In this talk, we share the experience of running Kafka as a stateful Kubernetes workload in a cloud-native environment using the open-source Koperator and the tools we built to automate the whole lifecycle. While running a stateful service...
May 27, 2022•17 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Running a database on Kubernetes with persistent storage is relatively easy but when it comes to performance it won’t match local NVMes. This talk will show you how to set up the local NVMes for Kubernetes, how to handle the application and cluster lifecycle in a safe manner and share our experience with running ScyllaDB with local NVMes on different Kubernetes cloud providers. Tomas l...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Redis is a widely used open-source in-memory data store and cache that has become a key component in the development of scalable microservice systems. While all of the main cloud providers provide fully managed Redis services (Amazon ElastiCache, Azure Cache for Redis, and GCP Memorystore), it may also be simply implemented in Kubernetes if you require additional control over the Redis...
May 27, 2022•8 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Accidental PVC delete or namespace delete can cause the Persistent Volume to get deleted. Such volumes lose their data and the stateful applications lose their state. By the use of Persistent Volume TrashCan, users can get a grace period to undo such unintended delete operation. The deleted Persistent Volumes are staged for delayed deletes. They continue to live even after being delete...
May 27, 2022•11 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The container storage interface (CSI) is a contract between different container orchestrators (Kubernetes, Nomad, etc) and storage plugins. This contract is a set of gRPC services for provisioning, utilizing, and snapshotting storage volumes. In this talk, we will focus on one aspect of the CSI spec: Volume Snapshots. The talk will cover volume snapshots on Kubernetes CSI, with a deep ...
May 27, 2022•11 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The ability to extend Kubernetes with Custom Resource Definitions and respective controllers has led to the OperatorSDK, which became the de facto standard for data service automation on Kubernetes. There are countless operator implementations available, and new operators are being released on a daily basis. Organizations managing hundreds of Kubernetes clusters for dozens of developer...
May 27, 2022•15 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) FoundationDB is an open-source distributed transactional Key-Value store that is used by multiple companies like Apple, Snowflake and VMWare Tanzu (previously Wavefront). This talk will cover the design of the FoundationDB operator and lessons learned from operating FoundationDB on Kubernetes. We will discuss some of the missing pieces in Kubernetes to make it easier to operate Foundat...
May 27, 2022•9 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) It is still challenging to run Apache Spark and other big data processing workload on Kubernetes, especially in large scale. People need to address various issues like resource isolation, queuing, and cost efficiency. This session will share details about those challenges and how to address them. We will also present a convenient (one-click) way to deploy Apache Spark on Kubernetes, an...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Data doesn’t magically appear in our data centers. There are usually several phases and several storage locations along its journey throughout your organization. New architectural patterns, such as microservices, and new technology, such as Kubernetes are changing how we can think about and manage the large volumes of data coming at us. In this talk we will begin by quickly introducing...
May 27, 2022•14 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes comes with a lot of useful features like Volumes and StatefulSets, which make running stateful workloads simple. Interestingly, when combined with the right tools, these features can make Kubernetes very valuable for developers wanting to run massive production databases in development! This is exactly what was seen at "Extendi". The developers at Extendi deal with a large a...
May 27, 2022•14 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Cloud-native applications comprise various components, including data services, storage systems, and related Kubernetes objects. Each component requires its own data protection tools, strategy, and domain expertise. A robust solution aligned with business requirements often involves complex workflows. What if there was a way to coordinate the implementation of these workflows while opt...
May 27, 2022•14 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) It only looks simple to run databases in Kubernetes. In fact, it is too many things needed to be considered before running any database in Kubernetes. Failover and traffic switching, replication and data consistency/loss after failover, upgrades, DB and node-level configuration, CNI, backups, monitoring, etc. After this talk, you will have a complete list of questions that should be ch...
May 27, 2022•15 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) How can you keep your data secure and how can you transfer them on a secure way? You will learn to encrypt your data, that you can use them Kubernetes based in a Multi Cloud environment. Sarah Julia Kriesch is working as a DevOps Consultant for Accenture in Germany. She has studied Computer Science at the TH Nürnberg and worked as a Linux System Administrator/ System Engineer for an IS...
May 27, 2022•7 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes is turning into a preferred platform for real-time analytic app that crunch billions of events per day and return insights in seconds. In this talk we'll introduce the standard analytic app design pattern of fast event streams coupled with low-latency data warehouses, using open source projects. We'll then walk through deploying the pipeline on Kubernetes from ingest to end ...
May 27, 2022•16 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Graph databases are the fastest growing data store in the world. According to Gartner, the application of graph processing and graph DBMSs will grow at 100 percent annually through 2022 to continuously accelerate data preparation and enable more complex and adaptive data science. However, it is often difficult for data and analytics professionals to distinguish between different implem...
May 27, 2022•20 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) With the advent of microservices in Kubernetes, individual developer teams now manage their own data, middleware, and databases. Automated tests and CI/CD pipelines have to be revisited to include these new requirements. This session will discuss and demonstrate how to use Kustomize and Tekton to provide Kube-Native automated workflows taking into account new parameters such as databas...
May 27, 2022•17 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The storage topology in vogue seems to cycle every few years. Internal storage is followed by centralized Storage Area Networks only to be superseded by one-size-fits-all Hyperconverged models - until scalability constraints led to distributed storage. Then comes NVMe, offering blistering speeds that all of these storage stacks struggle with. Kubernetes inspires Container Attached Stor...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Many ML pipelines depend on shared filesystems for input, output and intermediate data storage. Standards such as CSI have made it possible for applications in Kubernetes to access a variety of data storage systems. Yet, data scientists still have to deal with low-level details of data access in order to execute their pipelines in Kubernetes. Datashim is a framework that manages the li...
May 27, 2022•16 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Apache Kafka is more than just a messaging broker. It has a rich ecosystem of different components. There are connectors for importing and exporting data, different stream processing libraries, schema registries and a lot more. This talk will show how to use it to read data from social networks such as Twitter, process them and use machine learning to analyze them. And all of it will b...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) What does Kubernetes provide that allows us to reduce the complexity of Apache Cassandra while making it better suited for cloud native deployments? That was the question we started with as we began a mission to bring Cassandra closer to Kubernetes and eliminate the redundancy. Many great open source databases have been adapted to run on Kubernetes, without relying on the deep ecosyste...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Managing stateful workloads in a containerized environment has always been a concern. However, as Kubernetes developed, the whole community worked hard to bring stateful workloads to meet the needs of their enterprise users. As a result, Kubernetes introduced StatefulSets which supports stateful workloads since Kubernetes version 1.9. Users of Kubernetes now can use stateful applications...
May 27, 2022•10 min
https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Come here about our experience scaling Cassandra on EKS to over 1000 nodes and 20 million transactions per second. This session will cover the lessons learned, successes, failures, and tools used to get there. Usability is Matt’s mission. He has worked with Federal, Fortune 500, and small businesses to help collect, mine and interact with data. When solving a problem, Mr. Overstreet sy...
May 27, 2022•7 min
https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK In this talk I share my personal experience where when I was working for a client the company had a hard time innovating and did not have a lot of faith in their software/infrastructure engineers. From a 'distance' I saw a lot of potential where they already had knowledgable engineers but lacked the experience to get started as a company but also as individual engineers. I created a community at that client, filled with th...
May 12, 2022•1 hr 1 min•Season 1Ep. 131
https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK What if databases only ran in a single place? That would be useless. But it's what we get with most database-as-a-service offerings in the public cloud. Kubernetes has the potential to change the game and let users run complex database anywhere they want. This talk tells the story of how we built a portable ClickHouse cloud service on Kubernetes. It runs on AWS and GCP, of course. Even better, we're working on managing dat...
May 04, 2022•1 hr 5 min•Season 1Ep. 130
