I'm ready, I'm ready, no move across this time, unfortunately. Excellent, that's a moment we get here, yeah, we will, we will, hello there, hello everyone and welcome to I'm so excited to the 10th episode, okay, episode of Topics, DevOps Topics and today we are going to talk about serverless on AWS, on Amazon, yeah, AWS Lambda. As we usually start, I'm not ready for the first question, I guess you don't know what it is, right?
No, no idea, no idea, it's no idea, yeah, yeah, all random, okay, so what's the first thing that comes up to your mind when I say serverless on AWS, all right, let me reverse the question, all right, it's been 10 episodes, I think it's time I'll reverse it to you, what's the first thing, when I just say out there, AWS serverless, Lambda is whatever, what's the first thing you're thinking of?
Well, I think when I think of serverless, I think of somebody else's computing machine that is allocated to me because he's kind enough to give me some time, that's what I think when I hear serverless, you know, which is the Cloud joke, right, it's not, there's no cloud, it's just someone else's computer, exactly, which is not completely false, but also not completely true. So let's get to the bottom, what does it actually mean?
When you run just a random workload, if I would want to use my application, usually the standard way to go is just to log into my AWS account, obviously any cloud provider, today we'll be focusing on AWS, but it works for any of them, I would log in create an easy to instance, which is a server, I don't know, SSH in, install something work, it's not my server, I'm leasing the time, I'm paying for the hours or actually seconds today that
it's working and I can turn it off at any time, I'm paying on demand price for something that is now running on my network. When I say my network, my VPC, right, by default, not accessible from anywhere to anywhere and I can do whatever I want. When we talk about Lambda's, and by the way, I'm going to use Lambda's in Fargate, not interchangeably, but they kind of work on the same platform, we'll touch on a bit on that in a moment.
So when you run at Lambda, you're speaking of two different things, one, in terms of what you said, you said someone else's computer, so that's completely right and it's not only someone else's computer, it's also someone else's network. And by that I mean that Lambda's have the wrong VPC on AWS, you're not exposed to them as a customer, you just send your workload and it runs on someone else's network with an internet access, by default, that's what the Lambda is.
In terms of the user experience, what you actually do is you write some kind of code, you have different run times for Go, Python, Node.js, Rust, many, many, many different languages and you send the code usually wrapped in a zip file and AWS does something with it, right? They run it in the cloud on their own VPC, it's accessible to the internet, fine, everything works. Do you want to go under the hood a little bit or are we pushing it too much?
I think you told me about AWS is under the hood of Lambda, privately, so maybe you should show it with everyone else, so what's under the hood under AWS Lambda, because I think most people say, hmm, containers, so maybe they think of container D, maybe they think of the Docker runtime or whatever, but how does AWS do that, you know, how do they learn our Lambda functions?
Yeah, so I was wondering, I was recently interviewed somewhere and I asked the, it was another interviewee and we spoke about it and he was working in Google and he said that in Google, he's pretty sure, which was very disnot sure, but he's pretty sure that it's work under the hood, Borg is Kubernetes. That's what he said that Google functions are working on top. If anyone is listening to that and can correct us or confirm, I'd be happy to hear.
In terms of Lambda, AWS are actually real estate, it's open source, it's a framework that's called Firecracker written in Rust, pretty fast, and the idea is running lightweight KVM's, KVM's are just those micro virtual machines that run on top of the kernel.
They're kind of like containers, but they're different, I think, in the sense that the kernel isn't sure because what you want with Lambda's, and by the way, again, when I speak of Lambda's, I also speak about the way the instantiate Fargate containers, Fargate are your way of running containers, not on nodes that you manage on someone else's nodes, which is AWS's.
So Firecracker is a framework to do it really, really fast in a lightweight manner, but also keep you in an isolated environment because obviously you don't want your Lambda cache or anything that's proprietary to your company ever looking to some places. So that's the way, that's Firecracker, and that's it, you ship your code, you send a zip file and runs, that's the basics of it. The basics of Lambda functions, yep, although you also touched a bit of Fargate.
So let me add a bit about the Lambda functions, okay, and I want you to intervene meanwhile, okay, ready? All right, so you said that Lambda functions, when they run, they run publicly and they have access to the internet, blah, blah, blah, blah, so I'll give you a use case, and I'm sure you'll be like, yeah, yeah, of course, of course. So I give you a use case where I used Lambda in VPC, you know, that visual Lambda in VPC?
To be honest, that's, I think usually that's most of the use cases, that's exactly the important point here, people just run it and they say, oh wait, but I need access to my databases and my secrets management and probably another workload that's running privately, what happens now? So I guess that's for even or even, okay, I'll give you another use here, well, I can say weird, but maybe an edge case, but it might happen to some other people.
So let's see, you use some third party provider and that third party provider tells you, okay, I'm going to accept only connections from these IP addresses and then you say, okay, I want to run my application in a Lambda function. So if the Lambda function has a public IP address, you cannot, you know, you cannot constrain it to a specific set of IP addresses. So what do you do? How can Lambda function have a specific IP address?
What can I do to whitelist my Lambda function in that third party provider? And the answer was provided, you know, one minute before, it was the Lambda in VPC, because if you provide it with maybe an at gateway, so it can go outside through that not gateway, exactly, okay, exactly, so exactly.
Let's double click on that because that's actually a very interesting topic, exactly what you said when you read to run privately for all the different wizards that we both shared just a moment ago, you need to have your Lambda running in a private network, maybe you don't want it exposed to the internet, maybe you do, but through a not get weight, so that's your option, you're using an ad for that.
And that used to be a problem because, okay, I have more often than not, you're running a lot of Lambda. It's not like one, two, three, because it's function, so it's very distributed and every little component of your system, and that's the idea should be separated into its own business logic that's running in its own function.
But then when you have 100 Lambas, 100 different functions, which is not a lot, I mean, it's a relative manner, but it's not too much, 100 functions, and maybe you want 99 of them or 90 of them connected to the VPC, because they need access to the database, or a secrets management. And if you're using a secret manager that sits privately, not something else that ships the secrets into your environment variables, maybe all of your Lambda's actually need access to the VPC, and then what do you do?
Because each of them would be assigned with network access, which one takes time, and two, you have a soft, I think it's a soft limitation in AWS and how many of these you can actually provision, so you can get to that limit. If you have trying to connect 100 Lambda's or 1000 Lambda's with each having network access into your VPC, it's a problem.
And not long ago by saying not long, I think, two years or something like that, but somewhere in the recent two years, AWS kind of improved the scheme, and they released your way of having as many Lambda's as you want connect to your VPC through a shared entry point.
So this network access, the card that they're using in order to connect, can be shared among, I don't know if all of the Lambda's, but at least a lot of them, and that improves both the speed, latency, everything that comes around that. So that's a huge improvement in that sense. I think that's basically it. That's what I have to say about it before. Before we even proceed with that, so let's break it down to, let's say we provide it now to use case.
So you said we can use Lambda in VPC to access privately sources like my database, for example, if you want to access Unity MongoDB, local host of whatever from a Lambda function to your database, that would be a great idea to use the VPC or whatever, oh, I don't know, private IP address or whatever, or if you want your Lambda function to have a specific IP address so that a third party would be able to whitelist that, that's Lambda VPC would do that.
But let's also talk about, I don't know, what's the weirdest thing that happened to you in your used Lambda VPC? What do you think is the first, the first L, the two encountered, the first time ever that you wanted to do that, okay? What were your issues? Because I had a very, very common issue and I think we should talk about that. So what was your first issue when you, when you used the email? There were so many. It's hard for me to pick.
I think we'll, after you share what you think, we'll touch on a bit about considerations and what you need to understand when going to Lambda, because as a developer coming to Lambda is a different mindset, both in terms of concurrency, how I handle requests, a lot of those things. Let's wait with that for a second. The biggest issue for me, that was a few years back, all of a sudden, by the way, same thing for Fargate, there's nowhere to SSH to, the environment is not yours anymore.
You can't handle the workload, there's no, no, there's nothing, there's something that runs somewhere, you're actually not sure where, even if it is connected to the VPC, you can't really see what's going on under the hood. Which as operations engineers, at least that's worked, we were always used to write, I can, it's a save, read the logs, connect to the instance, maybe replace it, do something, none of that, no more exists. So, yeah.
So, what was yours, you only get logs, that you only get after the Faggler, and we said you see the Faggot in session, and the ones that you export, if you didn't, your problem, you need to run again. And every second is built. So, yeah, there's not like a hot debugging, like you said, like you cannot SSH to something that is actually a way to hack it, but you're essentially right, there's no way of doing that.
Okay, so, to me, the first pitfall, I mean, ever that was for creating a lambda in VPC, is that you're like, you want to do that, and you don't realize why it doesn't work. And then you realize, as you said, AWS attaches an elastic network interface, an ENI, to each lambda function, or maybe shares it, either way, you must provide the proper permissions to the lambda function, to create that ENI.
So, if you forget to provide your lambda function, to create that ENI and attach it to the lambda function, that's only relevant to a lambda in VPC, yeah? So, if you forget about it, you don't understand why it doesn't work. So, you tick the lambda in VPC in your serverless framework, YAML, or whatever framework you use, maybe AWS SAM, or I don't know. And you don't understand why it doesn't work.
So, make sure you provide the proper permissions to create and attach the ENI for the lambda functions before you use the lambda in VPC. That's my main pitfall for starting to use, you know, when I started using the lambda in VPC. So, do you also have some great segue? Sorry? Do you also have something like that, something very annoying? No, it's actually solution to something very annoying, you know? No, no, no. Well, kind of. So, this is a great segue to speaking about frameworks.
I think most of the developers, I can say that we do in my company, with a framework there's more than one, there's, I think, the first one was serverless.com, that's based on node. You have SAM by AWS and you have all kinds of other really great platforms which are wrappers around serverless, because what you just mentioned is actually relatively simple. You'd see what the error is, you'd go fix it. If you try configure that by hand.
Let alone configure the endpoints coming in from an AWS at least API gateway, you'd be lost in a hell of configurations and metrics and things you're unsure. And frameworks for the rescue, if you use something like the serverless framework serverless.com, that would let you describe everything you want in this YAML format and ship it away.
And it would under the hood, it creates cloud formation stacks, but essentially it makes all the connections and all the wiring, it handles everything you need. And everything you need, it's permissions, like you said, the actual instantiation of the function. But on top of that, you need to be able to access the function, right? You need to expose it to customers, that would be through something like an API gateway. That needs a configuration of its own, maybe use open API, maybe you don't.
You need to configure the REST APIs, the protocols that are coming in, getting out, maybe you want to do some kind of digestion of the request, maybe manipulation of parameters, headers you allow, et cetera, et cetera, et cetera, et cetera, et cetera. It's tons, a ton of configurations and you have frameworks that help a lot with that. So these are the frameworks and the standard solution that I think most people use today.
I can also talk about, maybe we can talk about the authorization authentication, because if service framework has also a great integration with, if you use AWS API gateway, version one, not version two. So if you want to use the Coppinito authorizer, okay, in case you're using AWS Coppinito, which is AWS's, you know, authentication solution, okay, serverless authentication solution.
So if you want to use that, you need to use the version one, again, not version, not the HTTP API gateway, but the less API gateway, and then you can use the Coppinito authorizer, and then you can authenticate users with AWS Coppinito to your Lambda functions, you know, which is cool, you know, it's very, very via the API gateway. It's interesting, because we started using that and then up the way to a different provider, but yeah, everything in AWS. Why did you move?
Okay, so let's talk about that. Why did you move from Coppinito API gateway? You can't go into it, okay, okay, okay, okay, okay, do you want to zoom out a little bit and talk about considerations and general approaches to Lambda, because I think for developers mainly, and everyone's starting to work with it, there are few things you need to consider.
So I think my goal here is to speak about what you said, and we'll talk about it in a second, and the other thing would be maybe talk a bit about local development, you know, so once you switch that, we'll move to local development, so I'll go ahead and talk about considerations for using serverless at all, okay, so let's talk about the application considerations.
When you use Lambda, the idea is every time a request is incoming from the outside world, whether that's, it doesn't have to be a request, right, Lambda can be triggered.
It has, when you'll first set up AWS Lambda, you'll see something is incoming, like visually, you have the Lambda, it's doing something, maybe ship's logs to cloud watch, maybe I don't know, puts a message in SQS, but something's triggering it, that can be an API gateway request, it can be a new message on SQS, it can be a file on S3, there are many ways to trigger Lambda, so when you, each time something happens, whatever kind of event
it would be, more often than not, again, in our case, that would be API gateway, an incoming request triggers one Lambda, if you have concurrency, and you want 100 requests, incoming at the same time, that means you have 100 Lambda, so Lambda is kind of your concurrency solution, whereas in native applications, or the standard way, you'd handle it yourself, some kind of concurrency model, some kinds of aggregation through a messaging system, whatever.
So Lambda handles that for you as a platform, 100 incoming requests mean 100 Lambda, okay, and now we're getting into this world of cold and hot starts. So essentially, we said Lambda runs on the firecracker platform, and when an incoming request is coming in, it instantiates an environment, and this environment would be some kind of container that holds your data, your function, your code, everything you need, runs it inside, and then waits.
In Amazon, I think it's something like 30 seconds, and if you have a subsequent request coming in again, AWS don't have to reset the environment for you, because it's already standing there and waiting, and then you can use it another time, and the starting time, obviously, is very fast, because the container or the KVM is already in the air waiting for you, and that would be called a hot start, and your goal is to utilize as many hot starts as you can, right?
Cold start is slow, if everything is cold started on Lambda, I wouldn't say probably doing something wrong, but you should take it into consideration, because it's hurting your performance. So that's something you want to use. Ways to do that. First of all, obviously, applicative, your functions need to be small, quick, fast, and they don't want to do things. You don't want to pull cues, right?
You don't want to use a polling loop for cues, you don't want to wait for stuff, you don't want to wait for HTTP responses, everything you do needs to instantly finish its job. If you need to use a cue, that's a standard way of operating Lambda. Use a cue to put a message, wait for it, grab a job, use that as trigger, don't wait for it. Don't pull for cues.
I hope I'm being clean with that, right, because if you use to all architecture where I'm sending a request, I'm waiting for a response, I'm waiting for something to happen. That's good for a process you manage. When you're built by the second, that could become very expensive. So what you want to do is offload the work to other resources. Now there are a few ways to do that in Amazon, when you talk about concurrency, there are two important things to remember.
One, you have the reserved concurrency, reserved capacity, reserved concurrency. I think it's reserved concurrency. In any case, you can say when you start an AWS Lambda, you have a pool of 1,000 Lambda's you can run concurrently, all right? And I can say this Lambda is very, very important. I want to reserve 100, 100 places. That would mean now the global pool is 900.
While the other Lambda's can use the shared pool of 900, if they reach that, they won't be able to touch the 100 that are reserved for a specific Lambda. But you need to use that with the thought that everyone else might get throttling or throttled because you have that one reserved. So if you use reservations on top of each and every Lambda, you'd pretty soon hit the capacity of 1,000, and then you need to speak to Amazon, increase it, and so on and so forth. One thing to consider.
The other one, we mentioned that you have a lot of cold starts. And if you have, you know, your picks. For example, I'm working in, I don't know, Virginia, and I know that at night, there are not a lot of requests, but during the day, there are tons of them. And I'm not utilizing, my system is not fast enough. You can pay for Amazon to keep a lot of, let's call them environments ready up in the air, all those KVM. So you can pay for, let's say, 100 KVM's ready to serve you.
That would be called a provision concurrency. And that's something you pay for because obviously your kind of Lambda environments are waiting up in the air for requests. You can do that hard coded. You can use AWS auto scaling for that. So when you have a lot of requests, you provision more and then decrease that, something to consider. Those are two important things. That's it. Okay. And I also, I just want to be clear about that, okay? So you may, you mainly focused on web applications, okay?
So the tips that you gave are mostly or even absolutely for, you know, explicitly for web applications. So if anyone out there is using Lambda functions for anything other than a web application, maybe like a backend API for some service, it runs once in a while or that's okay if it's a call start. You know, it's okay to have it with a call start because you don't need the, you know, fast spawning of Lambda functions. Unless you need the speed. So just consider that, have that in mind.
Yeah. So it's a very important. And also I'd say, I want to say also something where I guess it's not, it's not that important. So we can just move to the local development and if I remember that I'll just tell you about that again. Okay. So local development with Lambda functions, how do you like to develop your Lambda functions in Somel? Which tools do you use? I think that's one of the areas that I haven't perfected yet. So nothing fancy. I think you can think about it on your own.
I kind of use tools to mock services that I need from AWS, but it's not good enough yet. So I do try to run on containers locally. So I kind of mimic the isolate environment. And when I need stuff like Dynamo and EC2, we use a lot of AWS APIs. So to mock that, I use motto, which is the testing framework, the mock service for a bottle and another one that I forgot the name will find it later. That's familiar with maybe local stack.
Yeah. So actually I'm using local stack and I'm very happy with it, but I gotta say we're using the pro version. So we also have like most of the APIs that people don't have. So the free version, the open source version has a lot of APIs like I think S3 and I also think Palamato store and many others. But if you want to have, I think like the local API gateway, I'm not sure that it's available in the basic free version. So I don't want to commit to that.
But if you're using the pro version of local stack, so it's a great way to develop locally with AWS. So that's why my answer is very nice. That was my other solution, not using the pro version. So I have to check it out actually. That's interesting. I never thought of it. We're using the basic stuff to mimic that, but that's it, nothing beyond that. There's probably, there are probably ways to improve the process.
I didn't find them yet or maybe didn't invest time enough to find them, but that's it. Okay. So before we finish, I want to give you the last tip from my end about serverless. Okay. No, it's not something that I remember, okay, something that I just want to say about the tip. Okay. I also want you to think, well, I do that. I want you to think about your tip, okay.
I have, so when I create, I have it, so when I, when I create serverless function, serverless framework or serverless, whatever application, usually what I do is I separate the infrastructure from the applications. So for example, I create the API gateway and let's say Cloudfront distribution and maybe S3 bucket and all the shared resources. I love to call it like shared infra, okay.
And then I export those shared resources because it's like on AWS and CloudFormation, so you can export those resources with an ID, okay.
And then in my, in my application or maybe a lot of applications, I don't know, I can consume the API gateway ID, less API ID, I can consume the Cloudfront ID, I do an import on my application, on my, yeah, on my application and, and this way I'm using the shared infra and I don't like if, if I want to change something in the infra, I do it separately, you know, you break the logic. So it also applies to, let's see, multiple applications can use the same infra structure.
So again, it depends on your, on your use case, but basically it works a lot of time that you have a shared infra and then consume it in your application, which is when I say application, I mean the Lambda functions, okay. That's what I mean. So go ahead with your tip over for the tip because you said something, remember my one minute story last week. So now I have a 20 seconds. It was a two minute, two minute, two minute, two minute, two minute, oh, you want a minute.
How would I, okay, I don't know. Let me tell a 20 second story and then I owe you one in a 20 second, all right, 20 second story, when our company was first started, obviously they only worked with Lambda. So they started one function and then they needed a dynamo DB table. So they needed to create that, but there is no means of creating any infra structure in a high level.
There wasn't no separation, the separation, it was coupled to the Lambda function, yeah, no, not only it was coupled to the Lambda function, it was a global table that all of the next functions coming into life in the next six or seven months were using. And then one day we started to migrate stuff with your started creating more environments and nobody knew where this table sits or provisions it.
You can't repurvision it, you don't know where it is and worse than that, you someone deleted the Lambda and then the table went with it and all kinds of weird stuff. So I think your tip is even far more important than what I'm going to say, separate the environment, separate the infrastructure, this thing that can you go with the application. If your function is okay with deleting everything around the server LSEML, by all means, create the tables, create the SQS, create the SNS topics.
If you're creating something like a dynamo DB table, everyone are going to use later. You probably don't want to do that, but you don't even have to think about it, just don't create dynamo DB table with the serverless function unless it is something you're doing for a POC. So that's it. Moving on, one thing I want to say, Lambda is not a tool for everything. It's a good tool for a lot of things. If you end up, it's expensive, you're billed by the second, like we said multiple times.
If you find yourself running tons of lambdas and falling for cues, waiting for events, doing some things you shouldn't, obviously, a rearchitecture is probably recommended, but a lot of times Lambda isn't the best solution. You might want to consider containers, be farget containers, but something that is a long living workload. An example for that, we talked about a lot of cold starts parallel work. When you hit with a lot of parallel work, there's a lot of problems beyond cost and performance.
Because what happens is, imagine those lambdas are running in the VPC are connecting to a database, right? If you have like a case, we had tens of thousands of incoming requests, all running in parallel. Forget about the cold start. I'm speaking only about 10,000 KVM's trying to connect to the same database, which has no problem with the data, but holding tens of thousands of them. A lot of fun, because of incoming 10,000 requests are killing the database. That would think about that.
It's not only using that database, it's probably, not probably. I'm actually saying the truth, it's using Redis instance and it's using Elastic for search. There you go. You have a downstream of failure. You can only do so much with a proxing connection pool for different workloads. It's not that easy. Sometimes you can, sometimes you can't, so that's another consideration for Lambda.
It's probably not wise to use anywhere, but when you do think about the downstream services, it's not the Lambda living in an isolation world. That's it, I think. So to sum it, I'd say it's like, I'll do it with a cool tone, like, serverless, use it wisely. Just like that. So I think we've had enough for today, okay? I think I hope, right? It's like we're on the 29 minutes in one second or something. Just enough time for a cool project you've seen. Not enough time for me, only you today.
All right, only you. Fair enough. I'm a productivity geek about everything. One thing that I really like is managing my windows. When I say windows, like, windows on my screen, right? And I like having a hotkey that I'm going to go sub to windows. No, no, no, nothing like that. I'm actually talking about macOS and I'll explain. If you're an avid Linux user, you're probably heard of i3, which is a tiling window manager, pretty cool. It lets you organize everything on the screen.
If you stretch one thing, the other thing collapses or at least reduces its size. So there's one developer created two amazing tools. One I've been using for a long time is called SKHD. Look for it. And GitHub will put a link below. That's just a hotkey manager. You can use a hotkey for everything. Open my terminal, open my browser, close Slack, open Spotify, whatever you want. The other thing he created is called Yabai and weird name, but Yabai is a tiling window manager for macOS.
And when you combine both, you have a hotkey with tons of shortcuts. I want to move that window here. I want to move just my focus. I want to stretch that. I want to rotate everything, change in my workspace. It has hotkeys and shortcuts for everything and it's an incredible workflow. I only started working with it yesterday and I'm already having fun. So once it becomes muscle memory, not me having to mentally think about each keystroke, it will probably be even better.
And then you will show us a demo because it sounds interesting. All right. Done, build. OK. So, Ome, thank you for being here. And to everyone in the crowd, of course. Thank you for coming today. Thank you for coming today. Thank you for the crowd. Yeah. See you next week, Amigo. All right, guys. Bye bye.