I saw a lot of people doing was just assuming that micro services were mini web servers. And sure a mini web server with a small API and sort of database is a micro service, but it's horrible architecture. So I think the way to do micro services, the Tao of micro services, is not to be obsessed with the network and Kubernetes and all that other stuff. To think of them first as software components and all the other stuff is, is just a consequence to get Legos, to get
components. You can't have complex interfaces. I was very inspired by Unix pipes. So if you think about it, that's one of the most successful composer models ever built in the history of computing. And that's because the interface is really simple. It's just a stream of bytes from one process to another. Just because the network is unreliable doesn't mean a monolith is unreliable either. It's a fallacy to assume that you can build an error free
system. The seven network fallacies are 100% true, but you don't deal with them by trying to make the network infallible. You deal with them by accepting that system overall, even if it's a monolith, has a baseline error rate. And that's a business requirements issue. Hey guys, welcome back to another new episode of the Techni General Podcast. Today I have with me and author an interesting book, I must say, at the Tower of Micro Services. His name is Richard.
Roger Richard, welcome to the show. Henry, thanks for having me on. Yeah, let's start arguing about micro services I'm looking forward to right, because, you know, everybody agrees on what micro services are and how to do them. There's no controversy at all. I'm sure we we'll get to a lot of different perspectives about microservices and monolith and things like that.
But maybe let's start the show by asking about yourself, as if you can introduce, maybe telling us any highlights or turning points in your career that we all can learn from you. Yeah, I graduated in 97 and I studied mathematics and philosophy and by random luck the mathematics course at AC plus plus module. So I learned to code on VI, original VI on VT100, green screen terminals in the attic of the maths department. So of course I ended up in going
into web development, right? It was just the pre.com boom era. The website that I'm most proud of, the fastest one I ever built was in C pure C. It was a real estate website. I didn't know about malloc so everything was statically allocated which is why it was really fast, but literally every operation was a buffer overflow waiting to happen. But it remains the fastest website I've ever built. I spent the 1st 10 years of my career as a back office Java developer.
I pretty much went to no conferences, did almost no community stuff. I did read a lot of stuff, but I never really kind of engaged outside of what I was, what I was working on till I discovered Nojs and the Nojs community and kind of got out of my shell. I got into startups and the first successful startup that I was involved in that did exit after I left, unfortunately built the core tech as the CTO.
We built something like a Roku for mobile apps, but if you use JavaScript on the back end, use the Rhino engine for JavaScript because we wanted it to be easy and quick to, you know, kind of like almost like no code, nearly quick to build a back end for mobile apps and developed a love for JavaScript, which is a weird language, but I don't know, once you kind of get over all the strange stuff, yeah, I don't know, it just kind of clicked in
my brain. You know, I like the object prototype approach rather than the sort of hard classes. When I was doing Java, I read the Gang of Four design patterns book. I could tell you off by heart all the patterns, right? Visitor everything. And then when I discovered JavaScript, I realized you didn't have to actually use any of the patterns. You can just go pretty much functional.
So myself and three other guys set up a consultancy based around Node JS and you know, sometimes you can get lucky. People talk about product market fit and if you have to ask the question, do you have product market fit, you don't. No, we were a product company. We tried, but we were just a
consultancy. But Node was taking oath around that era since of 2012 and we were pretty much the biggest node consultancy in Europe. We started a few conferences, I started speaking at various conferences, that type of stuff and nothing took off. And within the space of two or three years, we had a big business, which is kind of crazy. So that was super fun, super
stressful. And then we discovered that JavaScript is not a great language for writing large enterprise systems, but our clients wanted us because it was the hype of the thing. So that's how we ended up using micro services or the idea of micro services, but also ended up with a kind of a weird take, which we can get to in a little bit. But basically the limitations, I mean, it's weird, isn't it, right.
I love the language JavaScript, but the limitations of JavaScript. And that's, I mean, I coded TypeScript now, but you got to remember JavaScript back then, ten years ago was pretty bad, right? It was kind of a sucky language. Once you've got any sort of code volume, you can't even refactor it. I mean, there's no type safety. Hopefully you wrote good unit tests, but they're probably tied to the implementation, so refactoring is almost impossible.
So out of the vadnais of JavaScript, we ended up using micro services to compensate. It's funny how things like that go with softer, isn't it? You have this idea that there's some sort of perfection or ideal way to build systems, but it's very, as the physicists say, it's very path dependent. So where do I get to? I, I, I have to get to today because you asked about the career history. So yeah, yeah, consulting is great.
But if anybody's working in consultancy, they know it's really hard work, tons of travel. And I'd still wanted to do a proper SAS, a proper traditional VC funded SAS. So because I've been doing a lot of conference speaking, I thought I'd spotted an opportunity in the event tech market, specifically a SAS solution to help people run booths and do sales league
capture and recruiting. And because I mean, I was speaking at like a couple of conferences a month, but organizing it all using a spreadsheet, which kind of sucked. So I sold out of the consultancy and sold out to the other partners and set up a SAS business, did the Angel round, took in a good sized round, built the product, everything was fantastic until March 2020. So we lost all our customers to go in the space of a month and
we had to pivot. Eventually, though, it took us about a year to figure out what to pivot into, but we realized that a key part of the market that we've been looking at was developer relations. And luckily, developer relations still has to happen even if there are no conferences. So yeah, we pivoted into that as a market, which is where we are now. However, it's never a simple straight line because of COVID, we never did a Series A. And you know, how do you survive?
So guess what? Consulting. So I tried, I tried to get away from consulting, ended up doing it again. Luckily we're kind of refocusing on the product now, but I ended up doing consulting again for a few years, which, yeah, really put the ideas around micro services to the test again. Because I guess the interesting thing was that although we didn't make the entire Eventech platform open source, about 80% of it was open source plug in
based. We'll get into why that approach was taken, but you know, that means that if you've built something that's based on a micro services architecture with open source components for one vertical event technology, it was a good litmus test for the adaptability of micro services. Can they be used in other consulting projects, in other domains? And I think we found that the answer was yes, but for a very particular reason. Now, I'll just tease it. We'll get into that later.
And that brings us up to today. So yeah, the one regret I have and if anyone here is if any of your listeners are a junior engineer, I would say that a couple of years in a much larger big tech company is probably worth doing at some point in your career. I nearly nearly worked for SAP when I lived in Germany, but decided to chose a start up instead. But I probably should have taken the job in SAP for a couple of years.
I mean, I, I've ended up having clients that are big companies, so I suppose I ended up understanding in that way. But even if you do intend to do a startup, I think it's important to work at a really big company to understand how they work so that you could sell to them. So that's I guess one, right? But apart from that, I've had lots of fun. I'm still coding, coding for 27 years. Not going to stop. Wow. Do not intend to stop.
Oh, that's the other thing. Yeah, you've got 20 good years from about 18 to 38. After that you've got to compensate with good design, because the old nighters don't work anymore and you can't code yourself out of complexity just with brute force because yeah, you lose IQ points, the brain starts degrading, unfortunately. So yeah, enjoy. Enjoy it while you can. Thank you for sharing your story, I think it's really interesting. Sounds like a roller coaster as well, right?
So you've been into so many different journeys. So definitely there are a lot of learnings. But today we'll focus more on the micro services, right? In which you have written this book called The Tao of Micro Micro Services. But first of all, I'm quite interested, like why do you choose this title? You know the Tao. Is there any specific meaning behind it? Yeah, yeah, it's interesting. When you write stuff, sometimes you you can sort of forget to
fully use a metaphor. And I'm currently doing the 2nd edition of the book, so hopefully I'll get to actually use the metaphor fully this time around. What it refers to is the fact that what I saw a lot of people doing was just assuming that micro services were mini web servers. And yes, OK, micro services is a very broad term, like everything in tech. And sure, a mini web server with a small API and some database is a micro service, but it's horrible architecture.
It's you know, all those articles that you read criticizing microservices for people saying, oh, we're going back to a monolith because we had a really bad experience. They are all legitimate criticisms. I came at microservices from a totally different place. I felt there was a different way how it's kind of the way I felt there was a different more philosophical way to approach it. Let's talk about that because I think that's where a lot of the potential has been missed.
It doesn't mean it it can't still be taken advantage of because the way we develop software is continuously evolving. So I worked as a researcher for a while. I had my very first startup failed. It's very important to have a failed first startup. I was completely broke.
So I ended up working as a researcher at a local university and ended up working in the telecom space on IP telephony, which involves something called the session initiation protocol SIP, which is kind of like HTTP except state based and has loads more verbs. And if you imagine 2 phone, one phone ringing another, then it's you're sending like ringing state messages that I don't know. Henry, have you ever played around with any IP?
So yeah, it's horribly complex. No, I wouldn't recommend it. In the startup that failed, I'd really gotten into, if you go back to the game, four design patterns, really got into the command pattern where you kind of encapsulate commands in the system. You can try to express all your business logic as commands right now. I mean, this is a very, very early version of something like CQRS, event based approaches, event sourcing perhaps, but very locked into the code, very hard coded.
My experience working with IV telephony led me to think about the power of pattern matching. So the problem with receiving different stateful messages for IV telephony is that you have to decide what to do with them. So you're trying to write a little state machine, but first you have to match what's going on. And the application we were building, you have to provide customization. So we just ended up writing a really, really simple pattern
matching algorithm. And so I mean, if you've written something, if you've used Haskell or something like that, you understand this idea of pattern matching. In fact, I think it's being introduced in a whole bunch of languages now. I think there's even something for JavaScript. It's literally the most brain dead simple approach thing you can think of where you're just look at a bag of properties and you're just matching against
some of them. I mean, you can get more complex regular expressions or matching interior properties or whatever, but it doesn't really matter. All that matters is that you have this bag of properties, which might be adjacent document or HTTP request, and you're just picking a few that you care about and then using that to do more business logic.
So when we got around to building Node JS applications, I kind of put these two ideas together and said, how do we turn the command pattern into a component model that uses pattern matching? So here's the thing about consulting, right? And have you ever worked worked as a consultant? Have you ever built websites for people? Yeah. OK, so here's the problem. It's the same job every time, really. OK.
Because what we do is consultancy is we turn Excel into administration interfaces, right? And there's always user management and permissions and projects and groups and there's always data ingestion, there's always report generation. Maybe some of them have a mapping application or something like that. But business applications are always the same. So if you're building a consultancy, you want to be able to reuse stuff because that makes you more efficient, better
job. And Node was very open source, so a lot of the MPM registry was growing exponentially. I know that has its own problems, right? The problem is a lot of those components were technical libraries. So stuff for like sending e-mail or reading an Excel file or database drivers or Orms, right? That type of thing. Yeah, they're components, but the problem is every one of them has like an API that you have to learn. And they're infrastructure.
They're not business logic. So let's say you're building a referral system. Your users can invite each other into the application. The logic is pretty much the same every time. So how do you build a referral component that you can reuse? That was the problem that we were facing. So we built the system where it was event based. The only interface between components was a Jason document, and the routing was based on pattern matching. Your system consists of.
This is before micro services, right? It's a single process application, just a single node process running on a bare metal server. This is the way we built things back in the day. I built an entire newspaper system using this description bottle, Apple Pay, everything. But the whole architecture of the system consisted of very isolated components. The only interface input and output was effectively Jason
documents. The routing to decide where a message went was based on pattern matching. And that meant that you could define a referral service, let's say by a set of messages, so an invitation that's happened, or generate an invitation code, or give this user a reward based on the fact that the invitation code has been redeemed by defining Jason documents. And you could have schemas if
you wanted, right? But essentially you didn't have to. So when a message comes into the system, you might have some tags that say this is a referral message and it goes to the referral component. So that was fun until about 2014 until like I said, things get too big, right? It's very hard to manage a system like that. It's hard to scale it. I mean, you can for sure, you can deploy multiple monoliths and put them behind load balancers and all that sort of
stuff. But then we started noticing things like serverless and Docker and micro services idea was kind of in the air and it was just an obvious next step to say, OK, well, why don't we take the referral component and turn it into a micro service and then we can deploy it independently and we could scale it independently and get all those benefits. But the important thing is it's not a mini web server. It's not a mini API endpoint. It only knows how to receive and send Jason documents.
That's it. So that's how we ended up with that. That's how we ended up in a place where we were using micro services. But with a component oriented approach. So I think the way to do micro services, the Tau of micro services is not to be obsessed with the network and Kubernetes and all that other stuff. To think of them first as software components and all and all the other stuff is is just a consequence. Right.
I think that this is a quite unique perspective because I think from all the journeys of micro service that I have followed, right, I think it has gone from ups and downs, for example, from monolith to micro service and then it goes back into monolith. These days some people call it modular monolith or whatever that is, right? But I think that there are still places where micro services
could actually work. And sometimes some people advocate things like domain driven design, bounded context to actually create a proper boundary. But the one that you just mentioned is actually to think of micro services in terms of three different things that I could gather from the book, right? The 1st is about designing the message first. So things like the Jason document you mentioned.
Second is the pattern matching, which is kind of like, you know, associate the kind of shape of the message and then you route it differently to different services. And the last thing is about additivity. You know, like think of it like you can compose, for example, if you want to work on some workflows, right? You compose that kind of workflow using different components or micro services. So maybe, yeah, this is I think the true maybe the tower of the way of you defining micro
services. So tell us about why these three are so important and why do you advocate this approach instead of, you know, some of the more popular way of designing micro service? Yeah, it comes back to this idea of components. First, you actually mentioned the most important one, which is composition. If you're designing object oriented system these days, you should prefer composition over inheritance, right?
So inheritance people realize that inherent, you know, basic inheritance is probably a bad idea. You end up tying yourself in knots, whereas composing these together is the way to go. So I mean, what's a component system? It's Lego, right? It's meant to be like you're 10 years old and you're building a spaceship with Lego and the things just click together. So how do you get to that point? How does that work?
Because everybody who's worked on large enterprise systems know that they're not Lego. The refactoring is horrible. I mean, everybody's had a iteration where you have to stop the world and refactor. So the composition element is really, really important. And I think the pattern matching gives that to you in with a really nice mental model. So first of all, I should say separately, the implementation details here don't matter. We built an open source
framework for doing this. But it's actually pretty small. You could do this in any language. You could literally write one from scratch for a Greenfield project or whatever. You don't particularly need to use a framework. It's it's more of a philosophy than than a framework. Pattern matching gives you composition because it allows you to easily go from the general to the specific. So one of my go to examples is
let's say user management. When you start building the system you might already have, you might already decide there's one type of user in the system and there's a little bit of logic to deal with that. So I don't know. Assigning a user to a project is 1 Jason document with a few tags that you could have and match up. But that only gets you through maybe two or three iterations because of course, systems don't have one type of user.
They have admin users and they have project managers and they have group permissions and all sorts of crazy stuff. So if you have an assigned user to project message, that is general, very simple, right? It's just adding an entry to a database table and setting up very basic permissions. That's your first implementation, but you might have additional business logic that has to happen.
If it's an admin user, there's a special value that has to go in or something into the database or they have to be put into a central registry. There's some sort of extra thing that has to happen. So in most systems, what you'd end up doing is adding a whole bunch of if statements in that logic, say, oh, if it's an admin user, do this extra stuff, but that's not composable. What you want to do is you want to leave the old code alone. Pretty much that just handles
normal users. Run that code and then also run extra code that handles the admin case. So what you can do is either before or after. And again, this is pretty simple to implement because all you're doing is you're saying, take a look at Jason documents and if a few of the properties match, do something. So I can say, OK, my properties match for adding a user to a project, add the user to the project.
Oh, there's also A tag that says this user is admin, so I'm also going to trigger the admin user message. You mentioned additivity, right? So that's additive. The old code stays the same and then here's the new code and you click it into the system like a Lego brick. And you can do that for all sorts of things. It's particularly cool for cross cutting concerns.
So one of my other favorite examples there is, let's say you want to have like creation time and an updated time in your database rows across the board for all of your data entities. Well, if you encode the data entity operations, you know, saving and loading as messages, then the pattern matching can intercept those messages and add in creation time and save time, auditing permissions checks, whatever, all sorts of fun stuff without that being visible in the business logic code.
The point is, you want to keep your business logic as business logic, right? Business logic shouldn't be setting creation times or worrying about permissions or any of that sort of stuff. So if you have these additive components, you can see how it's pretty easy to start saying, well, maybe they actually live on separate parts of the network and maybe the messages can flow over the network to make these things happen because it's all just pattern matched routing.
The business object code has no knowledge of whether something is on the network or not. So you can run the whole thing as a monolith. And this is how we develop, correct? We run the whole thing as a monolith locally. I mean, local development with real microservices. And I have done this and I have, you know, killed my machines like 30 Docker services and crazy stuff like that. These are not lessons that weren't learned without a lot of blood.
But you can run locally as a modular monolith because you've got your modules. And then our favorite deployment these days is split up the messages by domain and deploy them as Lambdas. And that's really cool because you don't have to worry about running things in separate service locally. You can use Terraforms, something like that to automate the deployment of the lambdas. So you you don't you're not worrying about working with lambdas and deploying stuff to debug and all that Lambda
development is kind of sucky. And then the other big thing that the component based approach gives you is really simple mocking. So anybody who's ever tried to build unit tests and build mocks is some, well, I don't know, have you have you heard of the banana monkey jungle problem? No, I haven't. So this is a, this is AI tried to imagine it in a very posh English accent, because I learned about this from a friend of mine who used to be a fusion scientist. He worked on the joint European
tourist. So he's very clever chap. So you must be right. So the problem with a lot of, and I mean, this is a big problem in Java. The problem with writing unit tests in a lot of systems is that you have all this dependency and you have to do dependency injection, all that some crazy stuff. So you need a banana to unit test the banana, right? But unfortunately to get the banana you need a monkey. And if you need a monkey, you have to get the whole jungle,
right? So you basically have to bootstrap your entire system to do a small unit test. Now, if the only way that your system is exposed is via Jason documents, then mocking them is really simple. You just hard code the response. You don't need all the rest of the infrastructure. You don't need to have mocking utilities that will build complex. AP is, and I think this is, it's a point I make in the book,
actually, I just remembered. To get Legos to get components, you can't have complex interfaces. You think about modern languages, they try to present the class as a component, but classes have methods, they have interfaces, they have static methods, they have member variables, they have annotations. Are they singletons? Are they? And it only gets worse, right? Because in order for the language to have power, the class construct has to have lots of different ways to interface with it.
So now you have to learn a new API each time, and someone has to model that whole idea of user types in a system that would have to be modelled. So now you've got to interface with it somehow, and it's very hard to compose together. I was very inspired by Unix pipes. So if you think about it, that's one of the most successful composer models ever built in the history of computing. And that's because the interface is really simple. It's just a stream of bytes from one process to another.
So that has its own problems. It's a little bit too simple. I don't recommend building. I mean, people have, right? But I don't recommend building enterprise systems with streams of bytes. So you need to just up the complexity a tiny little bit more, right? So Jason Documents, I used to be really big into XML, by the way. I've written XML parse results of fun stuff. It's cool, but it's so painful, which is why I like Jason so
much, right? Because it does make life a little bit easier, even though I could really do with having comments. But anyway, if you up the complexity just a tiny bit and you allow the messages that go between components to be Jason documents, and then you, you know, apply that leniency principle where Postal's law, I think it's called, where you're leaning too much, you accept and strict in what you omit. It's just powerful enough to
build enterprise systems. When you use pattern machines, it covers about 95% of use cases. For the 5% it doesn't. You always have to be able to expose the actual underlying API, right? Always have a get out of jail free card, but 95% is pretty good. I've kind of covered a lot of ground, but I'm trying to get the philosophy Yeah, in one body. So I have some follow up questions actually. So when we think about these messages, right, you keep mentioning about Jason documents
and things like that. Some people also these days have even driven architecture, right, even like modelling software as events. Is this message you are talking about the same as kind of like an event? From my understanding, it seems not. I mean like it's not fully 100%. And then how does this pattern matching work against the message?
Because when you talk about composability, I think it's kind of like easy to, how should I say easy to think about when you talk about function calls or maybe like Unix pipe. But when you have different micro services and you have different network calls and databases, how does this composability work? And not to mention the transaction as well. So maybe if you can maybe elaborate a little bit more on these two parts.
Yeah, so my concept of messages is a bit more general and I'll actually refer to the book because I did spend a bit of time in the book creating an architectural structure for thinking about it. So event sourcing is very much unidirectional flow and that's a great idea. And it's a core benefit is that writes go in One Direction and reads come from a different place.
It's a kind of a one way system. The real world is a little bit messier, especially since you might be dealing with legacy systems and all that sort of stuff. And a great many AP is are designed to be request response. So I identify kind of two axes that messages could operate on. So this is one of them is whether they're synchronous or asynchronous. So synchronous messages expect a response. You send a message and you you expect a response, you're waiting for a response.
So if I save something to the database, I expect a response confirming that it's saved. It gives me a new ID, something like that. Asynchronous messages are ones where you just emit them and you don't know, right? Somebody might act on them, you just don't know. And then the other one is whether they're consumed or not.
So you can think of a classic message queue where you're sitting there as a listener and the message comes from the queue and you take it and process it and then the message is deleted. Amazon simple queue service, something like that, right? Or you can think of something like Kafka where there might be
multiple readers. So one of the people that I found really inspiring in the early days, microservices, and if you go onto YouTube and look for his talks, they're absolutely fantastic, is a guy called Fred George who built a lot of the early micro services systems. One of the examples he gives is insurance quotes. So you're trying to do dynamic insurance quoting. So somebody types in their details and they give me quotes and the system could process for about 30 seconds.
So the way he implemented that system was that messages would go out to lots of different micro services that could handle different types of quotes and then they would all provide quotes and then there would be some business logic to rank them or something like that. And that's an example of a system where the messages are not, they're not consumed by single service, they're broadcast.
So when you're building a system, you can think of it in terms of whether the messages request response or whether it's more like event sourcing. So whatever way you get messages from micro service A to micro service B, whatever way the routing works, whatever the pattern matching works, give yourself that freedom because you will need it.
But I mean, sometimes the criticism that I get and something that sometimes I've heard people struggle with is that they think I'm hand waving when I'm saying our message routing is do whatever you like. So let me give you a very concrete example. So recently we built an AI chat bot for podcasts, right? It lets you kind of interrogate guests. It's kind of fun, just as a kind of fun side project. And that runs in Amazon, uses Simple Q service. And if you think about it,
there's all sorts of stuff. You have to download the RSS feed and then you have to download the audio. And then if you have transcripts, and then you have to chunk the transcript and then you have to put it in the vector database and do all the fun stuff, talk to the models to get the vectors. So it's very much batch processing flow. It's very much messages go from A to B through different lambdas.
And if you built lambdas and use SQS, the way that works is you have named topics and one Lambda will submit a message to a named topic and then another Lambda is listening to the named topic. So that doesn't fit my model because if you think about it, those named topics are hard coded. But I won't have a situation where I'm free and easy and I I'm not stuck with topics. I just want to send my message and it magically gets to the right place.
So in my my modular monolith that I'm building locally when I'm developing the system, I just say that my, let's say my audio downloader microservice will listen for messages that say here's the RSS. And then it gets the RSS and it can look for the audio file download. Which means that my component that downloads the RSS can emit that message. Everything happens in one
process. I have a tiny little algorithm that looks at the keys of a Jason object in memory and then makes a method call to the right place. So somehow I have to translate that to Lambda SQS architecture. Initially when we started doing it, stuff like that was moved into configuration. So you would say, OK, these patterns of messages, yeah, it's like when you have a hammer, everything is a male, right? These patterns of messages all
mapped to this topic queue. So I have an extra bit of code that or an extra plug in, let's say that I deploy, when I'm deploying to Amazon, that intercepts the messages. This is a common pattern, right? Intercepting. It's just pattern matching like anything else, except this time it doesn't do any business logic. This time it does transport. It'll transport the messages for you. And for a long time that configuration was done like any configuration system, right?
It was in the code you configure that and there are option configurations for these plug insurance. But one of the reasons that I decided to do a second edition of the book is model driven architecture. So this is giving anybody who's my age nightmares at this stage. So about 20 years ago, it was really, there was this a lot of hype around UML diagrams and you would literally architect the system as a blueprint and then all of the junior coders would just type it out basically, right?
Total failure because that's how my software actually works. But I think there was a kernel of an idea there that was really good, which is that. And sorry, then you can combine that with like domain driven design and things like that to say there has to be a higher level abstract description of the system that's above the code in some way. And I mean, we use that all the time these days, right? So YAML definitions for serverless deployments, Kubernetes infrastructure as code, right?
But you can have architecture as code as well, not diagrams, right? Not visual programming, because that never works. But you can essentially use all of the very robust configuration tools that we have these days. There's a lot of them, right? So take the ideas in Terraform or Palumi or things like that and create a type safe model of the system where you are listing your database tables and you're listing your services and you're listing your messages.
And then you can do things like say, oh, here's the Lambda deployment context. And in that context, these messages go to this topic in SQS and Bats. Now, deployment configuration completely separate from business logic. So I never have to, when I'm writing my podcast subscription code, my audio download code, I never have to care. I just don't care how they talk to each other. I just know I can take for granted that the message will get delivered.
And then you add in lots of the modern observability, like open telemetry for actually debugging that. And yeah, it's kind of fun. We've a lot of fun these days. Right, I can also see a few alternatives to pattern matching. I mean tell me if this is not correct, right? So you can use something like a server path routing, right? So where you have HTTP/A goes to
a service slash B to B service. Or you can also take the message bus approach right where you have some business rules so to speak, right in the message bus to actually route to different topics. Or maybe in like Rabbit MQ you have different channel and exchanges and things like that. Is that also the same kind of thing when you mention about pattern matching? Yeah, I have a very big hammer.
They're all nails. So the issue is the power that you get from keeping everything to pattern matching is that things like unit testing and mocking get much easier. Or especially if you have junior developers on the team, the only API that they interact with is essentially you can send messages or receive them. Now we do add a little ORM layer, right? Not Object Relational Mapper. That actually becomes messages. Anyway, we'll talk about that in a minute because that is quite powerful.
But when the junior developer is writing business logic code, all they should ever be doing is sending and receiving messages, loading and saving stuff to the database. That's it. So if those messages are coming in from Rabbit and Q or they're coming in from HTTP request or whatever, the routing should translate that first. So for example, on the HTTP side of things, what we've written for ourselves is little gateway plug insurance that will
translate. So locally, I'm old school, we still use the Node library Express for local development. So we have a little gateway that can translate Express inbound Http://requests into a Jason document. Yeah, there are, you know, a subset of requests where you need to be able to access the headers and all that sort of stuff to do both. But again, you can use the pattern routing because that's a cross cutting concern.
You can put it elsewhere and you can just restrict business logic to saying here's a Jason document. It happened to come in this case from HTTP. But you don't know, maybe it's a unit test, right? So you know, the way unit testing APIs is really painful because you have to simulate AP Http://requests. You know, there's a few people that do it really well, like Fastify. Yeah, I'm obviously using no JS examples here. Fastify is really good system for simulating inbound HTV requests.
But the issue there is to use that you have to commit to Fastify. The problem is if you're deploying to Lambdas, well there you're not really using Fastify. I mean, you could, but generally you just want to use the events that Amazon sends. So now you have a whole different system. So you have another little plug in that can translate Amazon events.
We recently had a scenario where we built a system on AWS, spent six months building the system, enterprise system to be told one month before deployment that corporate policy was that everything had to run on Azure. So you can imagine that in most scenarios that that's a total disaster. If you have used any Amazon AP is directly you're in trouble. But because we've modeled the whole system as messages, all we had to write and we already had some because we obviously built
other Azure systems. All we have to do was write things like a little gateway plug in for Azure Functions which have a slightly different inbound message for which by the way is adjacent document anyway, right? There's another good analogy for this approach which is AWS Eventbridge. So if you've used Eventbridge at all, it's actually very similar or Erlang OTP, similar style. These are all either influences or same basic idea.
So yeah, it is a very, very big hammer and I doubt advocating for a very specific approach to building large scale enterprise systems. But at the same time, it's all driven by this idea that you really want a composable component model and the only way to get there is to have a simplified API. So you always want to hide things like Rabbit in Q or the fact that it's actually be requested at the end of the day. One final thing I'll say on that is I'm very much aware of the seven fallacies.
Network architecture. And I know very well that what I'm advocating for is to pretend the network doesn't exist, but that's a different discussion. Yeah, I think the one that you mentioned like it's called transport independence in your book, right. So yeah, even though you use web server, HDP web server or you use message queue or you use some other mechanism, I think it doesn't matter as long as you.
So I'll translate that into messages and you use some kind of pattern matching to be able to route to different services. And I think this key about composability is something that maybe not many developers naturally think that way, because if you think about adding user types, most of the people will just refactor the code and tweak the portion and do if else and that kind of thing. But I think doing pattern matching maybe is like the functional paradigm as well, right?
The functional programming paradigm. 100% I mean, I stole the idea from Erlang totally, yeah. So I think these are definitely the heuristics that you advocate whenever you design micro services. And obviously in your book you also covered about model driven, you know, like modeling code. Yeah, yeah. Which, yeah, some people like it, some don't, but we probably won't. Diagrams. That's that's the secret. It's all code.
It's all code. Yeah. So maybe as part of this everlasting debate about micro service versus monolith, what's your take about monolith, by the way? So do you actually ban monolith at all in your software design, or do you actually find that monoliths still have a case? No.
So, you know, because we use a modular monolith effectively for local developments, it's a wonderful get out of jail card because that project that we're selling that where we had to move to Azure, we converted everything from Lambdas to Azure functions, set everything up. And in the last week, I turned out that the admin who could give us appropriate permissions was on holiday. So there was no way for us to get enough permissions to make the system work and go live by
the deadline. So our solution ended up being to run AVM and run the system as a monolith in production. And now there were some performance issues, right? Because this is where micro services are great for scaling because parts of the system can scale independently. So we did have to deal with some performance issues, but we had to get out of jail card and that we could, It was nothing to do with the software architecture, but the dynamics of the
politics. The project meant that the only way we could go live was to deploy monolith. So we still had that option. And I would not have had that option five years ago. I mean, this is when people say micro services are crazy and monoliths are easy. They're right. And when people say, you know, if you're building an NVP, maybe you should just build a monolith to begin with. And I actually agree, right? Just deploy it on a bare metal hexer server or something for
$10 a month. But wouldn't it be nice if it was a component based model that you could turn into lambdas or a big Kubernetes deployments without having to rewrite the whole thing? It doesn't have to be my specific approach to pattern matching. It doesn't have to be node JS. You just have to start from a place of saying that my system should be composed of components, and all the components should have the same, just like Legos, they should have the same interface.
And that's really where all the power comes from. Yeah, it's one of those silly debates that isn't real, but it comes from places like, I mean, I'll tell you, we were brought into a large enterprise client. This would be maybe 2016. Micro services are getting really popular and they built a whole load of mini web servers and they were running about 60
micro services. And the problem is that even though they've given all the developers really powerful laptops, as a developer you couldn't run the system locally. So every developer was also given a large Amazon instance to run the system so they could develop against it. And I mean, of course, if you come into a set up like that where you have a team of 50 developers that can't even run the system locally, of course you're going to cope with micro
services are utter nonsense. Why on earth would you do this to yourself? The only people that I've ever seen that have actually managed to make the mini web service thing work is Netflix. I mean, they had the resources and the brain power to make it work. If you look at their open source and all their tooling and all that sort of stuff, wow. I mean, it's amazing.
I mean, yeah, there's a fantastic way to control everything, but it kind of tells the story, doesn't it in, in that they needed all that brainpower and resources to make it work, you know, And I, as an advocate of micro services, you know, surely the easy Rd. is to use Netflix stuff. But I don't have a team of 20 Stanford graduates to make it all work. Not to mention running all those Netflix tools right? I think that might be quite a set of people. Right.
Exactly. I mean, I'm sure there are other people who've who've done a really good job of it. Netflix open source, I kind of tell, yeah, it's what's the Spider Man purse. With great power comes great responsibility. So I guess services are really cool and you can do really cool stuff. But as the Australians say, they can be a massive foot gun as well, right? So, so maybe let's go into what you mentioned just now about the network policy and also the
data. I think these two typically are the the two biggest things, right? So network is you can't assume network is always reliable and fast. And second thing is about data. When you have your data passing across multiple services, there's no atomicity anymore, right? Everything is like an eventual consistency and how do you reconcile if there are failures in one part of the service into the other? Yes, yeah.
OK, so here's the thing. Just because the network is unreliable doesn't mean a monolith is unreliable either. You know, when I used to deploy monoliths in C and Java, stuff like that, VMS would run out of disk space because the logs have used up everything, right? Or memory leaks, right? I mean, when I used to run Java servers, but we used to just restart them every 24 hours because of memory leaks.
But the problem is if you just randomly restart a server at 4:00 AM in the morning, well, that's not 4:00 AM for everybody. You could be in the middle of transaction and they can just die. So if you walk into a supermarket, do you think that the prices are 100% accurate across every single label of every single product? No, there's actually in the USI think the law is the federal regulation is that they allow 2%
error rate. So the price that you see on the shelf is incorrect up to 2% of the time. I think, and again, this is inspired by Erlang, you have to design your systems to accept and a baseline error rate of some kind because weird stuff is always going to happen. Yes, the network is going to make that a bit higher. Of course the trade off is always the trade off is that you
get scale. So you have to look at your business case and think about what the mitigations are going to be. If you're building a, a consumer app, you know that's to do with user generated content. I mean, how many times have you used Reddit and it doesn't load? It's like we try load again, right? So I mean, who cares? There was a network failure,
just reload. I've built banking apps and I can tell you that they're very careful with their mainframe COBOL systems that do the actual transactions. But I can also tell you that there are backroom staff that reconcile problems. I mean, have has your bank been 100% correct over your entire lifetime? You hear these stories of people who are like magically have $10,000 appear in their account, this type of stuff. So it's a fallacy or telcos and they're five nines up time.
I mean, have you ever had a cell phone call fail? It's a fallacy to assume that you can build an error free system. So then you have to look at the business case around what the mitigation is going to be and is that is it serious enough that you have to have a backroom staff overnight like banks fixing things? Do you solve it economically by compensating your users in some
way? Are you Google where your profit margins are so great that you know, what if your search results aren't that good anymore, it doesn't really matter, right? So yes, the never, I mean that, you know, the seven network fallacies are 100% true, but you don't deal with them by trying to make the network infallible. You deal with them by accepting that system overall, even if it's a monolith, has a baseline error rate. And that's a business requirements issue.
And sometimes it's just, I mean, for a lot of systems, it just doesn't matter. It's just hit reload in the browser and keep going. Especially a lot of back office enterprise systems, that's just the way they work. It's a little bit different for some types of consumer apps where a baseline error rate might mean that you get hundreds of support requests a day, right? So that's going to suck. So here's, I mean, here's the interesting thing that the modular component based approach
can do for you. If you're finding that this a micro service A and micro service B that are really codependent and you're getting an error rate due to network issues that's too high. Well, combine them into one deployment artifact, right? Turn those network polls into
back into function polls. Now you're, they're still modular because the the interface is still just the messages, but you may find that you're beautifully designed, partitioned, are domain driven architecture has 10% of cases that need to be optimized. And when you have to optimize code, the code just looks bad, right? So you stick them together and deploy them as one artifact. So that's the networking side of things. The data side of things is interesting.
So first of all, it's kind of similar in that if you actually look at the details of transactions in traditional database systems, if you actually look at what they guarantees they offer, they're not actually that good. If you look at the specific details, like if you look at Oracle, right, there's actually, I think 4, there's four
different transaction levels. You might see old data and you might end up having to use them because you need the throughput of your system or it might not matter. So designing a system where you need to do distributed transactions, yeah, that kind of sucks. It's a void if possible. Unfortunately, it may not be possible, especially if it's a large enterprise system, multiple teams. But hey, you know what, even in those scenarios, there's usually like a lot of different models.
So you still have the same problem, right? So then you're in enterprise service buses and reversing things and all that sort of stuff. You just use SAPSAP Solved this in the 1980s. No seriously. Like if you look at R3, if you look at the SAP system, it's like the perfect micro service deployment system. They have solved all this stuff. It's just you need like $2000 a day consultants to actually make
it work. So especially if you're not building mega scale systems, try very hard to keep transactions contained inside one service, one unit of deployment. And if you've got this modular architecture, then again, you can stick things together. The other thing which I advocate in the book is don't just go to transactions automatically. There's loads of other approaches you can use.
I mean, even sort of brute force things like having a batch process that will correct data that's gone slightly wrong. Reservations. So reservations are really cool. That's kind of a standard algorithm to prevent like double spending. I mean, I won't go into the specifics of it, but you're basically creating database entries that basically block other things from happening. But it doesn't need, it just needs atomic rights. You don't need transactions. I'm a little bit naughty when it
comes to the database. I have to say I don't mind micro services sharing databases or tables. I don't know where that idea came from, this kind of obsession with purity of the database per micro service.
I guess if I may provide my understanding, right, it's because of the leaky internal data model that if you use two different services sharing the same database, right, the other service could just say, oh, I see this table, I see this data, I'll just query it and, you know, think of it as if like, it's my data, right? But actually, yeah, yeah, yeah, yes. Yes, yeah. So the data becomes a secondary interface which is now needs to
be managed. I mean, I don't see how monoliths are immune to that problem either, right? OK, so that kind of leads me on to the way that we use this idea to represent data. So I don't know if you were aware of another argument going on in parallel between people who dislike ORMS and people who speak ORMS. You should have ORMS, and I've used both. Obviously my original C code was directly talking to my SQL and I've used Hibernate, sharded Hibernate for Java, which is
wow, wow, how are we going? OK, so here's the thing. I think one of the big learnings from the no SQL movement was that joins are overused. And almost always 90% of cases you really don't need to join tables. Now that does introduce certain inefficiencies for sure. But the nice thing about that is it keeps your business logic relatively simple because you have very strong ideas of specific entities, like there's a user and there's a shopping
cart and there's a line item. And if you denormalize, again, you have to add in maybe batch processes to correct things and reservations to keep everything consistent. But mostly if you denormalize and you don't use joins, mostly you're OK for most enterprise. And certainly if you're doing a consumer application for scale, you're going to need to denormalize anyway, leave the joins to the reporting engines
and things like that, right? So I mean we preferentially use things like Dynamo DB these days. You can't really to join some of that, right? Yeah. So one of the underlying things I find that where people are not enthusiastic about microservices is because they also tend to have to give up the idea that you can join tables and write custom SQL inside your application. Microservice component based approaches tend to push you in that direction.
That's a big thing to give up. But I think it also goes back to the idea of being inspired by Unix pipes, because simpler that you can make the shared interfaces, the less trouble
they give you. So even if you have shared tables, shared schemas, if people aren't doing joins mostly OK. And then if you only expose the data via messages anyway, if the schema drift or something like that, then you can add interception messages that will temporarily adjust until you can figured things out, right. So in the book, I give some examples of migration strategies where you deploy multiple services or translator services to handle those types of scenarios.
You know, they pay us good money because this stuff ain't easy. So I don't think there's any easy answers to that. I, I just think it's not as it's all about trade-offs, right. So my personal trade off is I don't really use joins apart from reporting. I'm still here. I live, I'm alive. I think the point where you make you use less joints. So I think in your book you mentioned about data duplication, right?
So maybe think of like instead of normalizing and doing joints, right, you can also duplicate data. Yes, it may still a little bit, but also like thinking in terms of eventual consistency, you won't get like a strong consistency anyway, right? So I think that's really, really interesting view as well. So I think we can't cover any. Everything interesting in your book? You can tell I could keep going another three hours.
So definitely for people who are interested in this new way of thinking, you know, the tower of micro services, I do recommend checking out Richard's book. I think it really provides you with a unique different views of how to tackle micro service problem. So Richard, as part of my last question, I would love to ask you this question I called the three technical leadership wisdom.
So if you can think of it just like an advice that you want to leave to ask the listeners here to learn from you, what would that be? You have to let the developers that work for you make their own mistakes and you have to create a culture of making mistakes blame free. So that means sometimes as a senior or the architect allowing code in the system that is maybe not the quality that you would
write yourself. And I think a great deal of stress, especially for new team leaders and new architects, comes from this tension where it's literally impossible to get everybody to write code to the I mean, maybe you think you're good, I don't know, right? But it's whatever your perception is. You're never going to get other developers to write code the way you want.
And I mean, that's why components are so important, because if if something is written one way inside a component is less, the blast radius is smaller. People talk a lot about empathy and they use the word empathy a lot, But I think in technical leadership, a lot of that empathy is showing leadership by accepting a diversity of technical perspectives and allowing people to maybe make a
mess. I'd much rather if the developer came to me and you hear stories of people who've like destroyed, deleted the production database, you've told like, I would hire that person without thinking. I wouldn't even look, I wouldn't even think about it. You know, I don't even want to see your code, right? You deleted production database. I'm going to give you a job. That experience is something that you can't Rep. I mean, I've deleted live websites. You can't replicate that level
of experience. But for an organization to appreciate it, you've got to have empathy, right, Instead of looking for that kind of blame culture. So yeah, go easy on the PRS. That would sum it up. Doesn't really matter. Anything else or that's all. I think we can also wrap up with that. I think that's the most important thing, right? So thank you so much for the wisdom. I think it's really powerful and beautiful, right?
I like the empathy side. Obviously, as many architect or tech lead, sometimes we are very strongly opinionated about certain stuff, be it architecture, design or even coding style, right? So sometimes it all doesn't really matter as long as you know the right outcome that you want to achieve. So Richard, if people want to follow you or they want to chat with you about all these different perspectives about microservices, is there a place
where they can find you online? I am RJRODGER, still on X Twitter. I'm started using Mastodon, things like that. Same username. But yeah, Twitter's still where it's at by the looks of it, for the moment for tech. And yeah, the microservice is British places like that. Right. Henry, thank you so much. It's been super fun. Yep, thank you so much for this time as well. So I hope people enjoy our discussion about Microsoft today all. Righty. Thank you. Take care. Bye bye.