In this episode, we cover the following topics: We discuss the features and limitations of serving files directly from S3. We then talk about how CloudFront can address many of S3's limitations. In particular, CloudFront is performant, inexpensive and allows us to use custom CNAMEs with TLS encryption. How to create a secure CloudFront distribution for files hosted in S3. What is OAI (Origin Access Identity), why we need it and how to set it up. We show how you can configure your CloudFront dist...
Jul 08, 2020•41 min•Ep. 110
In this episode, we cover the following topics: A common feature for web apps is image upload. And we all know the "best practices" for how to build this feature. But getting it right can be tricky. We start off by discussing the problem space, and what we want to solve. A key goal is to have a solution that is massively scalable while being cost-effective. We outline the general architecture of the solution, with separate techniques for handling image uploading and downloading. We then dive dee...
Jul 01, 2020•43 min•Ep. 109
Show Details Jon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache conclude their series on the birth of NoSQL and DynamoDB. They compare the NoSQL database, Leviathan, created by Chris’s startup in the late 1990s to today’s DynamoDB. A lot of things haven’t changed, even though technology has evolved. It’s cyclical. There are patterns and problems that continue to dominate. Some of the highlights of the show include: Reason for Creation of NoSQL Database: How to scale dat...
Apr 15, 2020•43 min•Ep. 108
Show Details What’s under the hood of Amazon’s DynamoDB? Jon Christensen and Chris Hickman of Kelsus continue their discussion on DynamoDB, specifically about it’s architecture and components. They utilize a presentation from re:Invent titled, Amazon DynamoDB Under the Hood: How we built a hyper-scale database . Some of the highlights of the show include: Partition keys and global secondary indexes determine how data is partitioned across a storage node; allows you to scale out, instead of up Un...
Apr 08, 2020•41 min•Ep. 107
Show Details Jon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache continue their discussion on the birth of NoSQL and DynamoDB. They examine DynamoDB’s architecture and popularity as a solution for Internet-scale databases. Some of the highlights of the show include: Challenges, evolution, and reasons associated with Internet-scale data DynamoDB has been around a long time, but people are finally using it DynamoDB and MongoDB are document or key value stores that offer sc...
Apr 01, 2020•30 min•Ep. 106
Show Details Jon Christensen and Rich Staats learn about Chris Hickman’s first venture-backed startup (circa 1998) and its goal to build a database for Internet-scale applications. His story highlights what software is all about – history repeating itself because technology/software is meant to solve problems via new tools, techniques, and bigger challenges at bigger scales. Some of the highlights of the show include: Why Chris left Microsoft and how much it cost him; yet, he has no regrets Chri...
Mar 25, 2020•33 min•Ep. 105
Chris Hickman and Jon Christensen of Kelsus and Rich Staats from Secret Stache offer a history lesson on the unique challenges of data at “Internet scale” that gave birth to NoSQL and DynamoDB. How did AWS get to where it is with DynamoDB? And, what is AWS doing now? Some of the highlights of the show include: Werner’s Worst day at Amazon: Database system crashes during Super Saver Shipping Amazon strives to prevent problems that it knows will happen again by realizing relational database manage...
Mar 18, 2020•33 min•Ep. 104
Original Show Notes: At the recent Gluecon event, a popular topic centered around how to prevent Cloud Lock-in. Chris Hickman and Jon Christensen of Kelsus and Rich Staats from Secret Stache discuss why you your time is better spent focusing on one cloud provider. If/when Cloud Lock-in becomes an issue, you will have the resources to deal with it. Some of the highlights of the show include: AWS Fargate is ‘serverless ECS’. You don’t need to manage your own cluster nodes. This sounds great, but w...
Mar 11, 2020•27 min•Ep. 103
Start with 39. The Birth of NoSQL and DynamoDB – Part 1 . If you like that one, finish the five part series. If you still want more? Ask me for advice. I'll tell you what are the next best ones at [email protected].
Mar 08, 2020•2 min
In this episode, we cover the following topics: Developing a system for automatically updating containers when secrets are updated is a two-part solution. First, we need to be notified when secrets are updated. Then, we need to trigger an action to update the ECS service. CloudWatch Events can be used to receive notifications when secrets are updated. We explain CloudWatch Events and its primary components: events, rules and targets. Event patterns are used to filter for the specific events that...
Mar 04, 2020•1 hr 8 min•Ep. 102
In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act III: Eventual consistency saves the web (circa early 2000s)". We explain eventual consistency and the motivation behind the philosophy. The BASE acronym stands for three key properties of a distributed system that utilizes eventual consistency. We define and explain these BASE attributes: Basically available Soft state Eventual consistenc...
Feb 26, 2020•49 min•Ep. 101
In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act II: The arrival of the Internet creates new challenges (circa 1998)". Problems with building large scale-out systems led to the "discovery" of the CAP theorem (by Eric Brewer of Inktomi). We explain what the CAP theorem postulates and break it down in understandable terms. The three properties of the CAP theorem are consistency, availabil...
Feb 19, 2020•45 min•Ep. 100
In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act I: Transaction processing (circa 1973)". We start with the motivation behind talking about database soup - why are ACID, CAP, and BASE important to understand? We define transaction processing and its origins. What exactly is a "transaction"? Transactions are governed by ACID semantics. We define and explain the four characteristics of th...
Feb 12, 2020•41 min•Ep. 99
Oh by the way, buy girl scout cookies from my daughter here! GIRL SCOUT COOKIEEEEES! In this episode, we cover the following topics: Technology is changing at an increasing rate, with a constant stream of new things to learn. We discuss how innovation has changed the rules of the game. "Life moves pretty fast. If you don't stop and look around once in a while, you could miss it." - Ferris Bueller Chris recounts a personal story that emphasizes the importance of continual learning and growth. Dur...
Feb 05, 2020•1 hr•Ep. 98
In this episode, we cover the following topics: We continue our discussion of microVMs with a look at Kata Containers. Kata Containers formed by the merger of two projects: Intel Clear Containers and Hyper runV. How does Kata Containers integrate with existing container tooling? How mature are Kata Containers - are they ready for production? We then take a look at unikernels, which take a dramatically different approach to solving the problem of providing high security with blazing performance. ...
Jan 29, 2020•57 min•Ep. 97
In this episode, we cover the following topics: We revisit a misunderstanding from last week's show to find out exactly what the Firecracker team means when they list "Single VM per Firecracker process" as a security benefit. We discuss what's next on the Firecracker product roadmap, with particular emphasis on support for snapshot/restore. We learn how AWS uses Firecracker in production today with AWS Lambda. AWS is currently working on updating Fargate to use Firecracker. We look at why they a...
Jan 22, 2020•1 hr 1 min•Ep. 96
In this episode, we cover the following topics: We review virtual machines (full virtualization) and their benefits and tradeoffs. We then revisit containers (OS-level virtualization) and briefly recap how they use OS kernel features to enable virtualization. Containers provide great performance and resource efficiency, but at the cost of losing strong isolation. Can we have the performance and efficiency benefits of containers but with the strong isolation of VMs? There are some promising techn...
Jan 15, 2020•1 hr 8 min•Ep. 95
In this episode, we cover the following topics: AWS offers not one, but two, managed services for secrets management. Systems Manager Parameter Store and AWS Secrets Manager have similar functionality, making it sometimes confusing to know which to use. We compare and contrast the two services to help guide your choice. The three types of sensitive data injection supported by Elastic Container Service (ECS). Understanding when sensitive data is injected into the container and how to handle updat...
Jan 08, 2020•47 min•Ep. 94
Support Mobycast -> https://glow.fm/mobycast <- In this episode, we cover the following topics: What is secrets management and why we need it for our cloud-native applications. Guidelines for best practices when handling secrets. We walkthrough a simple, roll-your-own approach to secrets management using encryption (KMS) and an object store (S3). Although this is a simple technique, it does provide a very secure (and auditable) approach to secrets handling. But, for most situtations, you'l...
Jan 01, 2020•56 min•Ep. 93
Support Mobycast -> https://glow.fm/mobycast <- In this episode, we explain how to move an existing ECS application to private subnets. We cover the following topics: We describe the existing application, which is a typical two-tier web application, with a web service fronted by an Application Load Balancer (ALB) and database hosted on MySQL using RDS. The current application is containerized and running under ECS. Everything (the load balancer, ECS cluster, RDS instance) is running on pub...
Dec 25, 2019•54 min•Ep. 92
Support Mobycast -> https://glow.fm/mobycast <- In this episode, we cover the following topics: Recap and analysis of Andy Jassy's keynote, including: The theme of this year's keynote is transformation , presented via 6 theme songs. "The hunger keeps on growing" (Dave Matthews Band, "Too Much") Storage performance is growing much faster than compute/memory (6x faster since 2012). This is enabling new innovations like AQUA for Redshift, making it 10x faster than any other cloud data warehou...
Dec 19, 2019•46 min•Ep. 91
Support Mobycast -> https://glow.fm/mobycast <- In this episode, we cover the following topics: re:Invent 2019 by the numbers: 65,000 attendees, 3,000+ sessions, 4 keynotes, 6 venues. Recap and analysis of Monday Night Live keynote with Peter DeSantis, including: What is high performance computing (HPC)? How AWS is reinventing the supercomputer. Why everyone should care about HPC, not just the scientists. How networking advancements are paving the way forward for cluster computing and enab...
Dec 18, 2019•52 min•Ep. 91
Support Mobycast https://glow.fm/mobycast In this episode, we cover the following topics: Before we get started, a CAVEAT. There are other (potentially BETTER) ways of accessing resources on private subnets. We'll talk about these (such as AWS Client VPN or AWS Systems Manager Session Manager) in future episodes. But a great choice (with the most flexibility/power) remains our current choice: a third-party software-only VPN solution. There are many options for third-party software VPNs, both com...
Dec 11, 2019•1 hr 2 min•Ep. 90
Support Mobycast https://glow.fm/mobycast Show Details In this episode, we cover the following topics: Subnet 101 Public subnets Used for public facing resources which allow inbound connections from the public Internet Private subnets What are they? Used for resources that should not be exposed to open Internet Do not allow direct access from open Internet Require use of network address translation (NAT) for egress-only Internet access Why use private subnets? Protect your cloud servers from scr...
Dec 04, 2019•58 min•Ep. 89
Support Mobycast https://glow.fm/mobycast In this episode, we cover the following topics: AWS re:Invent general overview December 2nd thru December 6th 2,500+ sessions, spread over 6 venues, spanning 2.5 miles of the Las Vegas Strip Discuss the 4 primary types of content and the pros/cons of each Sessions, chalk talks, workshops and builders sessions Our general observations of themes to expect this year Hint: Kubernetes is hot We point out some of the sessions we are particularly looking forwar...
Nov 27, 2019•1 hr 1 min•Ep. 88
Support Mobycast https://glow.fm/mobycast Show Details In this episode, we cover the following topics: Container networking ECS networking mode Configures the Docker networking mode to use for the containers in the task Specified as part of the task definition Valid values: none Containers do not have external connectivity and port mappings can't be specified in the container definition bridge Utilizes Docker's built-in virtual network which runs inside each container instance Containers on an i...
Nov 20, 2019•59 min•Ep. 87
Support Mobycast https://glow.fm/mobycast Links Techcrunch article Mirantis Docker End Song La Place by Iwa For a full transcription of this episode, please visit the episode webpage . We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: [email protected] Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast...
Nov 16, 2019•23 min
Support Mobycast https://glow.fm/mobycast In this episode, we cover the following topics: Identity and access management for ECS Primary roles ECS Container Instance IAM Role ecsInstanceRole IAM policy and role required by ECS agent to make ECS API calls on your behalf ECS Service Scheduler IAM Role ecsServiceRole ECS service scheduler makes calls to EC2 and ELB APIs on your behalf Register/deregister container instances with load balancers ECS Task Execution IAM Role ecsTaskExecutionRole Also u...
Nov 13, 2019•57 min•Ep. 86
Support Mobycast https://glow.fm/mobycast In this episode, we cover the following topics: Amazon Elastic Container Service (ECS) basics Orchestration system for containers Well integrated with all the other Amazon services – More bang for your buck ECS components Cluster Logical grouping of tasks or services For EC2 launch type, set of EC2 instances that are defined and managed by: Launch Configuration Auto Scale Group Service Allows you to run and maintain a specified number of instances of a t...
Nov 06, 2019•1 hr 5 min•Ep. 85
Support Mobycast https://glow.fm/mobycast In this episode, we cover the following topics: Container runtimes Responsible for: Setting up namespaces and cgroups for containers Running commands inside those namespaces and cgroups Types of runtimes Low-level Handles tasks related to containers such as: Creating a container Attaching a process to an existing container High-level Handles "high level" tasks such as: Image creation Image management Defers container tasks to "low level" runtime Containe...
Oct 30, 2019•57 min•Ep. 84
