How'd you like to listen to dot NetRocks with no ads? Easy? Become a patron For just five dollars a month, you get access to a private RSS feed where all the shows have no ads. Twenty dollars a month will get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, Carl and Richard, here with your twenty twenty four NDC schedule. We'll be at as many NDC conferences as possible this year, and you should consider attending no matter what. Ndcoslo is
happening June tenth through the fourteenth. Get your tickets at ndcoslo dot com. The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Early bird discount ends April twenty sixth. Tickets at Cphdevfest dot com. Ndcporto is happening October fourteenth through the eighteenth. The early bird discount ends June fourteenth. Tickets at Ndcporto dot com. We'll see you there, we hope. Hey, guess what it's dot net rocks. Well you knew that because you pushed the play
button. I'm Carl Franklin and I'm Richard Campbell, and we're gonna have a good show today. Barry O'Reilly is here. We're going to be talking about fragility, or anti fragility as he talks about it. But how are you, sir, up on the coast back home? I didn't power out it yesterday for a couple hours of my new ups rig. You know, you find every mistake. Yeah, right, So the network stayed up perfectly, but it's like, oh, you plug that monitor into the surge protected outlet,
not the power backed up. So over the course of the two hours, I had my head lamp on and I was running around making fixes on stuff so that when it all came back, everything would be better. That's so so like you, It's so yes, it's like, oh, powers out now, I'm busy. I was busy last week trying to come up with a solution to a problem that I couldn't believe was a problem. But that's my better no framework, So let's roll the music. Also, apartment,
what do you got all? Right? So everybody does authentication authorization in a myriad of ways in twenty twenty four, not one right way. That's for sure. Oh no. But you know, if you just select when you're when you're creating a Blazer application, if you select individual accounts, you know you're basically buying into the Microsoft sp net Core identity system and it and it wants a database, right, and you can create the template is really
good now. And I got to say thanks to Jeremy Lecknis and he's not working on that now, but he's one of the guys behind all the new UI in the Blazer template that all Razor pages now and it's all in there. You don't have to scaffold anything, well most of it. But you know, roles is something that seems like an antiquated thing, right because everybody's using claims and building their own claims and we got claims and servers and two
tokens and all that stuff. But there's still a case to be made for simple roles like I want to know if you know, Joe is in the admin role so that he can change things and somebody else isn't simple things like that, And when it turns out that those are just strings, they're just tags, and so it's very easy for you to they've really turned into claims effectively a role in a claim is not that different. Yeah, and in a role is a type of claim. Right. But here's the deal.
If you're doing a Blazer server application, you just add one line of code to that template in your program cs, you know, ad roles of identity role, then you're fine. You do that in a web assembly applications a little bit more complicated. Yeah, now, but there's but but in the template, there's support for roles in the you know, in the in the server side and the client side, things that talk to each other. It's
just that Oops. They took that out right before dot net eight went to you know, in October, right before dot net eight went live, because there were situations where you have, if you have multiple claims with the same name, they were interfering with each other, and instead of fixing that, they punted on it. And it's still an open issue. So essentially you don't have roles. Wow. So basically I figured out a workaround for it.
Wrote a component that's free, and it's an MIT license component called Blazer Authorized Roleview And it's a replacement for the authorized view component that you can use to you know, authorize markup based on a role, except that it works in a web assembly. Uh, and it's probably a workaround. They'll probably fix it or they'll come up with some other solution. Something will change for nine. Yeah, maybe hopefully. So that is it. This is show
eighteen ninety six. So if you go to eighteen ninety six dot pop, dot me, that will take you to my repo. And there's also a Blazer train about it. So that's what I got. Excellent solve problems. Cool. Who's talking to us, Richard crowded commentator Show eighteen ninety four, the one which is just recently with Carl Geittz talking about programming and speech, and this comment comes from Rob who says, Hey, I really appreciate your
coverage of artificial intelligence. It's positive and tamps down fear. As an instructor, I hear a lot of concerns by other instructors about how AI will affect students, mostly from the perspective that a students are using AI to cheat, but from a programmer's perspective. To me, it's just another tool, and it's a really cool tool, but a tool nonetheless, And the sooner as instructors that we know how to use this new tool in the context of whatever
job. We are teaching students to do the better. So I loaded up gethub copilot. I use it, and that has made me realize that, yes, it makes me faster, and yes the student could still figure out syntax more easily, but what's wrong with that? They still need to learn the syntax to complete the code. They still need to learn how to read the code, and they still need to understand how complex frameworks are structured and
how to properly and effectively use them. And most importantly, they can still look forward to hours and hours of WTF research the figure out how to solve complex problems that I give them. Here's zer kind of instructor. It seems like most of my day is WTF. Yes, a lot of WTF research, right, just turn on the light. That's got to be a hashtag man, and AI is likely to never be able to figure everything out. That's our job as human beings. You know. I was recently on a
different podcast. I was on Jeff Plermo's podcast, and I said, listen, you know your job's not to write code. Your job is to find solutions, right, and if there's a tool that helps you make they get to that solution faster, you should use it. That's the responsible thing to do exactly. So, Rob, thank you so much for your comment,
and a copy of music Coby is on its way to you. And if you'd like a copy of music Cobe, I write a comment on the website at dot at Rocks dot com or on the facebooks you publish every show there and even comment there at e reading on the show, we'll send you copy of music. Go bye music to code by still going strong. Stuff in use by thousands, I would say thousands of people based on my sales. Dude, it's in this room every day, like don't don't be crazy,
Like that's the truth. When I'm home and I got to work mostly writing, to be honest. Yeah, that's what's on Cool Blue. I love Blue. Blue puts me Blue puts me in the zone. Like click, that's the first one. Yeah, okay, Well, you can of course follow us on Twitter if you want. But the cool kids are hanging out. I'm mastered on these days. I'm at Carl Franklin at tech Hub dot
social, and I'm Rich Campbell at masterd dot social. And you can find all the ways to get in touch in the at Carl Franklin dot com All right, let's get to our guest who hasn't been on in sickx years Berry six years. Barry O'Reilly is the founder of Black Tulip Technology and creator of residuality theory. He has held chief architect roles at Microsoft's Western Europe Consultancy practice.
Among others. He's been Microsoft's IoT Tap lead for Western Europe and worldwide lead for Microsoft's Solution Architecture Community. He has also been a startup CTO and was founder of Sweden's Azure user Group. He's currently finishing up a PhD in complexity science and software design. I really want to know about complexity science and in particular how it helps me oncomplexity five. Yeah, welcome Berry, Welcome back. Yeah, heeers. It's been a long time. A lot,
a lot has happened. Last time I was on, I was talking about the con accept of anti fragility and software and how we can build a software that appeared to survive on its own even though it wasn't designed for things, and that all of those ideas came out of work you did at Microsoft trying to teach new architects. So how do we turn good developers into into great architects kind of thing, and that I wrote a paper, and that paper led to a lot of things, a lot of attention, and I decided
that I wanted to do things a little bit differently. So I went back to school and the things that I had noticed, the things that i'd written about, I said, you know, if you don't gather requirements, if you don't try to predict the future, if you just randomly mix stuff up and stress and application, you actually end up with an architecture that's better than what we normally do. And that's a very big statement, you know, and it's it's fun and it's and I realized that this is the way I've
been working for a very long time. I've given up on process and templates and all of these things, and and you know, we're we're all just making stuff up until it starts to spin, until the wheel starts to spin, and the hamster looks happy, right, And I decide, I decided, you know, this is a wild claim. And while it's entertaining for me to go to a conference, getting a stage and talk to people about these wild claims, wouldn't it be great if if they actually had some depth?
So I went back to university, and I said, I want to I want to do a PhD. I want to produce academic work that says this is real. This is a scientific this is a scientific fact, not just not just someone you know, because if you get up on stage in I conference, this an amazing thing happens. People believe you. Yeah, And and I wanted to, you know, make it a little bit more. I wanted to answer the hard questions. And it turns out that that
that's actually was an incredibly difficult thing to do. And I'm approaching the end of that process now. I've just released a little book on Lean pub that's going to be given away as a gift at some conferences across Europe in the next couple of weeks that talks about the ideas. And so for the last six years I've been burrowed away writing frantically and rewriting and testing and teaching people
how to do these things. And the basic idea that instead of using requirements, instead of trying to predict what's going to change and using patterns and encapsulation and all these methods we've learned over decades, you can simply start with an architecture that's really simple that you know is wrong. One example I use that, I say to architects, just what you're allowed one component and all of your data has to be handled in memory. You're not allowed to do anything
else. There's one way in, there's one way out. This is how we're going to solve the problem. And if they want to make a change to that architecture, they have to suggest something that will cause it to fall apart. And they say, wow, what happens if we read but then we lose the in memory data stores? Okay, well you can have a data store that's you know, find you deserve state, you've qualified. Yes, you've justified persistence. So let's do that, and and the same with
cues and everything else. And you build your architecture up in that way. And the funny thing is when you do this, when you stress your way to an architecture, you eventually reach a point, a magical point, where you can't seem to find a way to break it anymore. And everything that you come up with actually is survived by the architecture. And that's what I
talked about last time. And so as they've gone away and read an entire library of books, and complexity, sciences and philosophy and all sorts of weird and wonderful stuff. I've managed to boil this down to a simple theory that says, this is why this works. This is why there's an actual scientific reason why you can produce an architecture just by being incredibly negative. You know. I just want to point out that writers, you know, writers of
fiction do this all the time. Okay, they just have to sit down and start writing, even nonfiction, right, Richard. Yeah, sure, you sit down, you start writing, even if it sucks, even if you know it sucks, and you throw lots of it away, throw lots of it away, but it gets your brain going. You have at least somewhere to start. Yeah. And if you and if you've got somebody watching over your shoulder saying that's not right, you just want to say, shut
up, you go away. I'm working it out here, man. Yeah. And that's that's one of the things in these ideas that's a huge, a huge culture shock for I think for a lot of software developers. Because we come from computer science backgrounds and mathematics and stam we're used to having a formula, We're used to being able to say this is the correct way to do this, This is the structured way, this is this is how you rationally arrive at an answer. And this, this way of working is a
little bit wild. You're wrong for a very long time until suddenly you're right. And that's a very very different way of working. And I've shown when i've shown these ideas, for example, too very senior architects, people who've built big stuff in azure, and they've said, yes, this is what we do. We don't have these fancy words or this academic waffle, but we we this is how we think we're we're you know, negative. And so if you're an architect, you've ever been called negative, that's that's a
very good thing. It means you're doing right. So it's and so it's starting to I've started to tour these ideas. I've been at a lot of conferences. There's a lot of a lot of views coming on YouTube, kind of growing interest for the ideas, and so I've started to publish books on the subject, and so they're starting to gather p s and it's it's a
it's a pretty exciting time for these ideas. I mean, I think the key point you're making here is that you're not afraid to constantly revise the architecture. But I also appreciate, you know, the insight here, which is, don't start with an architecture like go minimal so that you're not tearing much down, you're only adding, yeah, exactly making changes. I tend to find I go into companies and help them when they're having some troubles with their
architecture. And sometimes you'll find people locked in combat over and oh, this should be an event driven architecture, or this should be an API driven architecture, and this is an exting I know, Yeah, how do you know?
Yeah, it's a lot of gut feeling, and not that there's anything wrong with gut feeling exact how most of us do our jobs, but it's it doesn't become a very concrete discussion when you're fighting about the differences between these two different patterns, and so boiling it down to something simple and saying, tell me, it's something in the environment that's going to push this architecture forces it to behave in this particular way, and and people they can come up
with stories about what might happen in the environment and what's that happening is that we're the way that we traditionally build architectures as we try to gather information from stakeholders by running around and saying, tell me your requirements, and they run away from us. Most of the time. They don't want to tell us, and they won't tell us because they don't really know what the future looks
like. And some people say that you can capture risk and you can use probability, or that some people believe they can predict what's going to change in an architecture and write all that down. That that's not something I've seen in my career that's possible for us to do in a complex kind of business environment. And so the solution of this problem is to just randomly simulate the environment.
And so even if you're just making stuff up and you say, what's what's going to happen in this in this system, and you make something up, you can still find a set of conditions in which your architecture breaks, and once you know where breaks, you can start to make it better. And it's so I was doing an example of this. I was talking to my professor yesterday and he said, well, you know, you're calling it a random simulation. If I just opened a dictionary and just pick a random
word, can you use this to find out something about the architecture? And I said, so, we were talking about building a system for the National Health Service in Great Britain and he opened the dictionary and he just pointed a random word and he said the word is shed. Can you use this to make this architecture better? And so I said, okay, let's just say that some doctors and nurses start offering health services in their garden sheds at the
bottom of their gardens. That's absolutely ridiculous, right, that's probably not going to happen in the near future. I'd never say never. And I said, I'm talking about medical services. So the question is what impact would that have in our architecture. Well, suddenly we're distributed. Our workforce is distributed in a way that it hasn't been distributed before, and that puts challenges on our architecture with updates, with sequences, with all of those things you know
that get nasty when you distribute your workforce. And I said, okay, and let's ask what's the path in our architecture. Can we move to that? Or is our the architecture that we've decided on from the start. Is it so stuck in it's in its assumption that it can't move to this?
Would it? Would we have to scrap it and start again. And then as you talk about it, you realize actually there there is a movement in the United Kingdom where people believe that I guess what you guys call the emergency room, what we would call A and E. There are people who want to have lots and lots and lots of small emergency rooms in town centers, you know, especially on a Saturday night in the UK, where people like to beat each other up and they need they need, they need quick access
to healthcare. Then and so, while the idea that doctors and nurses will start practicing in their garden sheds is clearly ridiculous. It's a random data point, it's a random story, there is actually what we call in complexity science and attractor there which is a distributed workforce that you start to talk about. And if you solve for the shed, you solve for this potential change in
policy. You don't have to implement it in your architecture. Now, what you're looking for is an analogy that can be triggered by any kind of word
yeah, and any kind of random now. And so the theory that I've worked on is that doing this in this random way on your architecture will actually give you a better architecture than asking for requirements or risks, or saying what's going to change and what's not going to change, and what I've done, then what I'll do when I release, if I pass my PhD after the summer is showed that this actually happens in a statistically significant way that when when
you do there's a way of measuring with success of these of these programs, and you can show that this happens almost every single time you use this method, you get an architecture that shows itself to be able to survive in conditions which it hasn't been designed for. I mean, I would argue that even when you do go in with an architectural vision initially, there's a halfway point in almost every project where that gets substantially revised where you just like, wait,
we now understand this problem much more deeply. We've done a bunch of testing. Maybe we've got to it may even be a V one, but you're a few features and you're like, we've thought about this wrong, and if you're lucky, you're able to make enough changes to be able to deliver on what the customer actually needed. Yeah, and that's one of the things that I talk about, is, you know, is that the way that software architecture is being done today at large companies is kind of in some kind
of gray zone or twilight zone in terms of what is architecture. And there's a younger generation of our of developers who don't want architects in the room. They'll know, they'll they'll light their torches and they'll chase you out, and they'll say go away with your lines and your boxes and your rules, and chase you back up into your ivory tar. And especially within the agile movement, there's a there's a distrust of architecture because people have been burned by these
big upfront plans. And and that means that a lot of times we'll will architecture is seen as something that's just going to emerge over time. And and and like you said, as you move through a project, you learn and you find things out, and suddenly one day the team gets to a point and this can be very far down the line where the architecture won't move anymore. So we'll we've just discovered that this is all wrong and we can't save
it with a simple refactory. They have to tear it all down. And then project management will come in and say, no, we're not going to tear it all down. We'll just keep building on top of this very shaky platform and we'll call it technical debt. So it sounds good, and that's where that's where we're going. And what I'm trying what what I'm trying to get architects to understand is that there are three major challenges to software architecture that
that we really don't know how to handle yet. And those three things are time, because your architecture we dross and if you look at someone's architecture, it's two dimensional diagram with lines and boxes, and that will not stay to the saame ever over time, and there's an idea that it should do. And then there's change. Over time, things are going to change, and that's the big problem. That's why time is nasty. And the problem with
that change is that we don't know what it is. You can't sit down and figure out what's going to change because it's a complex business environment and everything is moving all at once. So the market and the customers and the employees and your competitors, everyone's moving. You don't know what's in the future. You can definitely, if you go back to partner US, you can definitely say the database connection strengths, we know that's going to change. We can
encaptulate that will be okay if we do that. But you don't know what's going to happen in your market. And so what I find was that when we do architecture, architects talk to each other through two dimensional diagrams, but in their heads they carry around the picture. Everyone has different pictures of many potential futures, many fears and diets and things that could go wrong. And so what I wanted to do is to develop a way of designing architecture that
actually made it easy to capture. This architecture has or a life cycle over time with many changes, and we don't know what those are. And the concept then becomes that this random simulation that we do where we randomly stress the thing from any perspective from talking about sheds, then we can arrive at an architecture that has an expression of uncertainty and time and change in it. And then we can show in a very simple mathematical way that this architecture is more
likely to survive in conditions which it hasn't been built for. It's more likely to survive those requirements or new features that pop up out of nowhere, those changes in the market. And there's a body of evidence for this in the
complexity sciences. So we go back to the nineteen sixties. A biologist called Kaufman developed a model for how life evolved out of a bunch of amino acids, how they connect to each other, how they control each other, and at the end that you can get these very simple amino acids, and at the end of the day you have something insanely complex like humanity. And we
think, oh, it must have a master designer. Somebody say, must have architected all of this at the outset, Yes, And herein we're engaging in these practices where that's see, it is like very difficult to do. Yes, And but the key to it is the same as the way life evolved is to randomly stress things until they take on a structure. And if the structure is wrong, and you keep randomly stressing your architecture, it will die and you have but luckily we haven't built anything yet, so we can
change it. And what we end up with is we have this concept of a residue, and a residue is an architecture in one particular timeline we have this is our residue, and it's whatever is left over in that timeline, and we change our original structure so that in that timeline things look a little bit better for us, maybe not entirely solve everything, but just just movable,
just survivable. And what we end up with is describing an architecture as you start off with your original naive architecture, which is the first thing you came up with, which if we're honest, you know we all do, and and a whole bunch of these residues that represent the architecture in different timelines. And the trick then is to integrate all of those different architectures into one
single set of components. And we do that using some tools that we steal from the complexity sciences called incidence matrices, and we use these matrices to find out where where is there hidden coupling in what I've done that's going to stop this architecture from moving through its different timelines if it needs to. And those matrices. In the book, I describe seven trigger seven refactoring triggers that you can see just by drawing a simple matrix and put filled in it with ones
and zeros, and it can actually guide you to component boundaries. And we're way outside of object orientation or nouns and verbs. Now we're building our building, our component structure, our service structure based on the different timelines that might exist in the future of this architecture. And it's a completely different way of thinking. It sounds very very alien. Compare too. We gather requirements, we measure all the risks, we follow some object oriented principles or gang of
four or something that then we build the system. This is a much wilder way of doing architecture, and it takes people a few days of training to get into the idea and let go of all all the old stuff. And I'm thinking about things in this very different way, but it does produce fascinatingly, produces really robust, resilient architectures, and in a way that's empirically verifiable.
You can show that I've done something good with a little mathematical formula at the end of your work, and that changes everything because as an architect, for the first time, you can actually show I've done something useful, which
is hard for architects. I can see that if you approach the subject with a word like anti fragility, yeah, that people's first thought might be, oh, that requires a whole bunch of front thinking about, you know, predicting the future, when in fact it's exactly the opposite, isn't it It is, yes, it's we're letting go of this false belief that we can
predict the future. And I think that's one of the things that enterprise architecture and waterfall approaches try to sweep under the rug for generations that yeah, we can predict the future, we can build a scenario analysis that's realistic, and in this methodology we give up all of that. You've never been able to do it, We've never successfully done it. If you're working in a system where you can predict the future, then you probably don't need an architect.
It's everything. Everything's easy in that kind of system. I've never really worked with any of those, but you know, it's they may exist. I haven't seen one well and they're I mean, I've certainly worked in environments where they're building the same app over and over again, essentially, and so the template does make sense. But yeah, not every most people don't work like
that generally. If you need to build the same app over and over again, that app order exists and you shouldn't need to build it at all. Yeah, if you're building the same app over and over again, some SaaS company is going to come along and steal your lunch. And so if you're not building that SaaS company, then you're going to be in trouble. Eventually,
you're going to get sass. Yeah, but yeah, I mean in the end, now, recognizing the body of work that's in front of us when we're building new software, it's because it doesn't exist, which means we really cannot predict the future. So it makes far more sense for us to just explore the space over time and come up with solutions as we go. It feels to me like the way you're describing this now is that the technical debt we're talking about is assumptions made early that built stuff we never needed.
Yes, the reluctance to change it once we do realize it because of the commitments we made to it. Yes, that's very, very dangerous. You'll find the one way that architects combat this kind of thing in the past is that we'll pick a platform, we'll pick a pattern, and we say this is the way that we're doing this, and we're because we because we're very wise, and we've looked into the future, and we've decided this is how
you solve this problem, right. And what happens is that as soon as you put that architecture, rub it up against reality, it starts to break a little bit. And the way we talk about that breaking is we call it edge cases, and we say we can't possibly be wrong. So everything that doesn't work is just an edge case. It's just a tiny little distraction
on the edge of our of our knowledge. And that's where you get technical debt because you start building and hacking to keep those edge cases and you won't let go of the architecture. And so these ideas give us a chance to question those assumptions very very early. And the way that the method is set up, it gives everyone, developers, everyone and the team a chance to come in and push the architecture around and say, you know, is it
going to hold in this timeline. And one way that we've dealt with that in the past is we use probability and we say that's never ever going to happen, right, But by analyzing this from I'm always afraid of that kind
of certainty. Yeah, exactly. But one of the things that we've discovered is that if you take a bunch of things that are going to happen in the system, they don't have they don't have to be accurate because every happening is part of a class of happenings that will push the entire system to the CM kind of state. And if you solve for that state, then you solve for many, many, many things. And what we've discovered is that
there's a simple mathematical leverage. If you look at it from a complexity science perspective, a software application is never actually complex. It's a complicated, constrained, very ordered thing. We tell it exactly what to do, and it only does what we've told it to do, and sometimes what we think we told it to do is a little bit different, but it's very very ordered. The real complexity in the software system is in the people and the markets
wandering around changing their minds all the time. Generally don't have you know, class libraries that change their minds, not yet anyway. And and which is this kind of there's this simple software system has to live in this massively complex business environment, and there's such a huge difference between them that there's a mathematical relationship between the number of possible states in them, which means that you don't
have to capture every little bit of information about the complex state. And so those waterfall approaches to requirements were always wrong on a scientific level, the wrong way to approach things. And so you only need a sample of those two to configure your architecture. And that's why we use the random simulation to make that sample as broad as possible, rather than narrowed by by language or by engineering metaphors that we've imported. And Barry, I'm gonna interrupt for one moment
for this very important message, and we're back. It's done at Rocks. I'm Richard Campbell. That's Carl Franklin. Hey, talking to our friend Barry O'Reilly, who we see regularly at conferences but don't talk too near enough. And I want to push back a little on what you were saying in the first half too, because a lot of that sort of random simulation is like it still feels like could we just call that requirements gathering, you know,
but maybe we're testing it with a bit of code and so forth. This idea that we're going to catch all this early, I just don't know that I buy it, Like, I think you are going to find things later on, and the question is how tolerant are you to being able to adapt later on? Yeah, And the methodology builds on that. So one of the most important concepts is that we don't know what's going to happen in the
future. We are always going to get surprised. And if you look at a business environ a business environment has tens of thousands and hundreds of thousands of people in it. It has technology, it has markets, it has politics, all of these things and they're all moving all at once. There's no
way that you can predict the movement of all of these things. And so the biologist Kaufman showed that when you have many, many, uncountable number of things connected to each other, you don't actually have to predict every tiny little state of every tiny little piece of a system. You just have to predict the presence of a certain number of things called attractors, and so a large complex system. You don't learn a large complex system by grabbing every individual and
understanding their needs and their wants and their habits and their peculiarities. You observe the system as a set of attractors that these things all influence each other and push each other into And so the way that you need to work as an architect is not to identify everything, but to identify as many attractors as possible. The moving parts will exactly and once once you expose your architecture to those
attractors, you start to reach something which Kaufman the biologists called criticality. And criticality is the property of a system where when you start to see that it survives things that it hasn't been designed for. And so a very good example is humanity or a human being. We survive things all the time that we don't even know are there. Our immune systems survive things that they haven't seen before because of the way they're configured. And Kaufman has this thing called the
NK model that explains it very well. And you have n is the number of components that you have in a system, and K is the number of links, and by optimizing for N and K in a system, you improve the system's ability to survive things that it hasn't been designed for. In software engineering, we've known this for a very very long time. We know that if you allow the number of components in your architecture to explode, and going to be harder to kill them all, but it's also going to be extremely
hard to manage them all if you've got lots and lots of them. And as you allow the number of connections between those components to grow, it becomes incredibly difficult to maintee and because you have to traceure your messages as they move between thousands of different channels. And Kaufman discovered that this exists in biological systems
as well. He calls it the NK model, and he noticed that as you tweak certain things the number of components, the number of links, or the way they interact with each other, by constraining how often or when they interact with each other, then you can control the number of attractors in a
system. And he notes that criticality is the property that you reach when a system arrives at the perfect balance of these things which we call the edge of chaos and complexity theory, because isn't the corollary also true that if we have too few components, we also ended with a lot of inertia because it becomes difficult to change them. Yes, and it becomes very easy to kill the system because there's only a few components, and if you take out one,
the whole falls apart. And so there's something we've always known this right as software engineers. There's something in between the monolith and the micro service, and it's never quite the same thing in different projects. But you have to find that line, that magic balance, and that magic balance can be fined through this random simulation, through stressing the system until you see that it stands up
on its own. And so instead of architecture being I'm going to design and build this beautiful cathedral, architecture becomes a little more like Bambie on the ice. Right, We're gonna slip slide, but eventually we're going to stand up and everything's going to be okay. This is absolutely fascinating to me. I'm just drinking in everything that you're saying here. And so you have do you have paper? You said you had a book or do you have papers other
than your book or is this all in your book? So it's all in the book, which is on Lean pub I have a link to it in the show notes. Yeah, awesome. The book is based on a series of seven academic papers that I wrote over the since we last talked, over the last six years. And what I find was I got a lot of feedback from academic people who said this is really good, and from a lot of complexity science people who said this is really good, and from a lot
of software developers who said, this makes no sense to us whatsoever. They're removing the cheese man. Yeah, and I've built a whole career around introducing complexity to the beginning of a project. Why are you ruining this for me with you exactly? And so what I what I did then? So I'm
working with the conference. The conference is called the Domain Driven Design Europe, and I've spoken there a few times on these ideas, and they called me last year and said, hey, we want to give a little book out as a present to all our attendees. Soul you could you write us a
little book? So I wrote the book, and I sat down and said I have to make this something that that's you know, more in the language that software engineers are going to grasp rather than all these fancy complexity terms from biology and philosophy and all of these things. And so that book has has just been released on Lean pub and it's it's much much easier to understand. And there's a second book in the works coming that's more on the philosophy.
So I ended up, you know, digging, going all the way back to the Presocratics and saying, well, this is where we went wrong. About three years ago in ancient Greece, a guy made a decision and we've all been suffering ever since. And so there's a fair body of work out there, and there's going to be you know, a PhD thesis published in the coming months, and there's going to be, you know, eventually a
bigger book. Just I mean not to dive straight into the academics of this, but part of what makes this methodology work is that software isn't tangible, That rebuilding the temple is a button press away, yes, you know, or accepting a pull request away. We did a lot of planning when building in stone, because getting it wrong takes a long time to clean up and replace. It's very difficult to go from straw to wood to brick where it's just not in software. Yeah, the analogy that I use in the book
is the manufacturing of cars. So and I say, you know, that's a very planned process, and it has to be it has to all work, everything has to fit together. And that's a kind of engineering. And whenever software engineers back in the fifties and sixties, we're looking for credibility, they wanted to say we're engineers too. So we borrowed a lot of the
engineering metaphor from industries like car manufacturing. And what I say in the book is, imagine what it would look like to be the designer of a car if gravity wasn't constant, if aluminium constantly changed its properties, if rubber melted and reformed randomly. Because that's what our stakeholders do, because we're building on their ideas, their opinions, and so using the old school kind of engineering
way of thinking about the world doesn't work for us. Software is so connected and so malleable, so connected to our ideas, and our ideas are fluid in a way that physical substances just aren't there. We don't have the equivalent of the laws of physics to constrain us. No't we can do anything. Anything can happen in a software project. And as soon as anything does happen, the rigid, sort of stiff software, the mechanical part of what we
do falls apart, very very quickly. And that's what I describe as as the fundamental issue in software engineering, that things are time changing, uncertainty are driving what we do. And we've been spending all our time trying to nail it down and make it certain right when we design, and most of the time we've been lying. And one of the things I reference in the book which most people haven't discovered this, and people have been emailing me saying this
is absolutely hilarious. But David Parnas wrote a paper in nineteen eighty six called a Rational Design Process Hi and Why to feake It? And so this is one of our is what this is one of our finding fathers of software engineering, and he says it's impuls in this paper, it's impossible to have a rational, structured way to design software. It just never happens the same way
twice. We're making everything up as we go along. It's pure mayhem, and he turns round in the paper and says, but there's a lot of value in pretending to management that we work in a rational and structured way. And so basically this article is a call to all software engineers to just lie. Why just make it look like it's engineering. And that's absolutely fascinating. Well, because it makes it makes the mortals comfortable, it does. Yeah
right, I mean that's really what it comes down to. And so he knew this forty years ago, and so we've been lying to management ever since about how how unpredictable and unstealable our methods and our processes are. And you'll find that in a lot of the approaches that we use, they promise certainty, they promise stability, they promise a repeatable process. If you look at things like TOGAF and things like see If that we have nowadays, it's all
about repeatability and everything being concrete and measured and well understood. And our world as architects simply isn't like that. Yeah, wow, Mind Blown Partners is still alive. He's actually in Victoria, Like I'm almost looking across at where he lives. Yeah, he's in his eighties. Like, yeah, man, I had some email contact with him a few years ago, and yeah,
we talked a little bit. So I pointed out that maybe, you know, maybe the problem that we've had is not that things change, but that we don't know what's going to change, which, well know, I've been wondering the past few years that the inertia of our systems have been pretty stable, and so it's felt more and more engineering recently. But at the same time, you know, I spent a lot of time looking outward. We're right for disruption. Yeah, you know, the smartphone's going to change,
likely to augmented reality. These language these non deterministic language models represent a whole new X paradigm, Like there's a bunch of funny things that could happen here, and boy, oh boy, all your plans can be blown up pretty good. Yeah. I think one of the things that's happening is we're getting better and better at what we do at the at the software, the rigid ordered part of what we do. We're getting better at delivering it. We know that we know how to test it, we know how to deploy
much more often, We're getting better at so many of these things. CLOUDE is this amazing collection of architectural patterns that makes everyone's life easier. But at the same time, if you look out into the real world, people who are doing business and executives and companies and governments, they're finding the world that we live in harder and harder to manage because they don't know what's coming down the pipeline or if anything's coming down the pipeline, and that makes their world
difficulty. Management loves predictability. Yeah, at the same time, we've sold all of the stuff is predictable when reality what is it should be allowing for extreme flexibility. Being able to pay for your your compute by the minute makes you more flexible. Being able to automate the pipeline deploy software means you can iterate more quickly. These are all elements of getting to where what you're describing
Barry, which is you don't have to predict the future. You can keep adjusting to the present as it comes at you because all of this infrastructure makes it very easy to do. So. Yeah, it's the question, you know, when a customer asks you, do you know how to you know? Can you do this? Not do you know? But can you do X? And most of the time I answer is yes, even if I've never done it before, because I trust in my ability to be able to figure it out and get it done. And so that is the certainty that
I have that I believe in myself. I believe in my team. Yes, and I know that we can do this because we have the ability to test things out, to look things up and see where the mistakes have been made and figure it out. Yes. And this is This is a big conversation I have with architects when I'm teaching them, And a lot of times teaching architects is more like therapy than actually teaching them anything. Sure well, one of the things I try to get them to understand is to say,
why were you successful in your last project? And a lot of architects will lean back and and say because I use this framework or this pattern or this tool. I was like, no, it's you. You are the success in your projects. The way that you think, the flexibility of thought that's required to build good architectures is what makes it succeed, and that that exists
only in you and the people around you. It doesn't exist in a free A framework boils away that human essence that makes it possible and that's why one of the one of the best forms of feedback that I get is when I teach these classes or I teach these workshops, and I have some senior architects who's built mad stuff that I respect, who says, this is how we
actually think, this is what we do. That's how I know that I've hit the kneel on the head because what I started out, what I set out to do, was to answer the question, how do I turn a developer into an architect? What do we teach them? Because it's definitely not too gap, and it's definitely not requirements, and it's definitely not lines and
boxes. And what I've done is I've turned that negativity of senior architects into a two that we can do so that we're not dismissed anymore, and so it'll be quiet negative, Nancy. We just want to get this built, and we can say, well, actually, this is the process. And it turns out that when you start to work in this random way, it's so much more fun. And it's one of the coolest things to watch at the workshops is when people start building systems by stressing them, by making mad
stuff up, by picking words out of a dictionary. Then the fun that it starts to happen is it's really really cool, and at the end of the day you get a verifiable architecture out of the work. I also wonder, I mean, there's a psychological aspect of this. By doing random things, by not, you know, doing all this careful planning, you're not
upset when your software breaks, like you break it all the time. Yeah, so it's I mean, you take away that stigma and so, you know, and you externalize it. It's like, oh, we did this thing, we got this result. This is not personal. This is not you've failed. This is we were experimenting. We battered the software, the software got battered. Good on us. Now let's try something else exactly.
This reminds me of a situation. I was actually lucky enough to be hired as a session musician on an album that was recorded at Levon Helms Studio and would stuck right, and so I was hanging around most of the you know, all I had to do is play one solo, right, and they finally got to me on the second day. I hang around all day the first day, second day, about three o'clock. All right, Carl you're
up. And I told them right at the outset, guys, it's going to take me about ten nine or ten takes to really nail this solo. And they looked at me like I was crazy, Like what you can't just like you know, and I said, no, I really have to play it a few times. I have to feel it to get some ideas, and you know, by the by the tenth take, I swear to God it's going to be magic. Turns out I nailed it. It was like
ninth take boom, that was the one everybody said yes. And it's because I understood myself and I understood what my you know, abilities are and how I can get to it. And so at the end of the day, I wasn't sure if I should have divulged that or I should have just done it. You know that if I hadn't said that, maybe they would have been angry with the fact, like, like, you're wasting our time, you know, you should just come in nail it. I don't know,
but it makes me. It just goes back to that whole thing with having a belief in your abilities and your ability to figure stuff out. Right. It don't look at it as an unknown look at it is what usually happens when I don't know something? And how long does it usually take me to figure it out? Yeah? And I think we have an attitude in our
industry which I call sigalism. And as you know, when you watch a Steven Siegal movie and he goes into you know, he goes into a bar full of bad guys and there's like fifty of them in there, and they all rush at him and he beats everyone up and throws them out through windows and things. He comes out of He comes back out of the bar without a scratch on him. He has a tiny little bit of dust on his
lapel which he brushes off and then he goes he drives off again. And you're thinking, in terms of plot, why would you go in there? That makes no sense. But in in tech we tend to think that, you know, we need to know everything. I need to go in and nail this on the first take. I need to be seen at a meeting. I need to be able to answer all of these questions about how this particular as your component works, and you know, I have to know this
stuff. And that leads to a sort of entrenched behavior when we're gathering requirements both in us who won't admit that we don't know something, and it bleeds onto our steakholders, and they're caught like a rabbit in the headlights. They have to say something or else look like they don't know what they're doing. And what they say then, in those circumstances is very often they're just making
it up to get us to go away. Whereas this kind of this kind of analysis allows us to take what they say and stress it and push it around and find things outside of it that we haven't understood. And it takes all that for us on the psychological side of things. It takes a lot of pressure off steakholders as well, once they realize that, you know, so, the question I asked to a steakholder isn't what are your requirements? Anymore? It's what keeps you up at night? What are you free it
off? What could go wrong here? And they'll happily talk about that because they're under no pressure to be present size. And the worst thing we can do to our stakeholders is forced them to be precise in their language or precise in their description of an unknowable future. We've heard and We've heard this before with this, and I've certainly had this experience that you don't interrogate like you're
a lawyer, like you're preparing for the lawsuit of this failed project. Let me get statements from you that hold me I'm no longer liable for this thing failing. Yeah. I usually say that we have a fantastic if you're a consultant or you know, an architectal large company, we have a fantastic business model, and that is tell me what you want, and I know it's wrong, and I'm going to build it until we find out that it's wrong, at which point I'm going to blame you because you didn't tell me,
and you have to pay me a giein to do the same thing. And it's this perfect business model where we have no responsibility for what we build, and this is an attempt to break out of that and change that or change that mold. I'm also impressed against the idea that there's there's nothing in the Edge Manifesto against any of this either. Arguably, this is what real agile was about, was lots of communication, minimizing what you were writing, responding
to change. It's just that it got well folks pushed back on that sort of sort of agile button mindset, right, that it keep water falling anyway. Yeah, I think that going back to without getting too philosophical, but there was a split in our think in three thousand years ago in Greece, and we chose the path of order and structure and abstraction, and we stayed
on that path for thousands of years. And it was good that we did because that gave us computers, but it also has some weaknesses that causes us to think and in a particular way. When we look at social systems, we try to treat them like manageable ordered machines. And the agile movement was actually a huge philosophical event from that perspective because they said, hey, we don't know what's going on. We can't actually write all this time. That's
a huge philosophical step. It's a huge challenge to our industry well, and arguably said can't know right, like it is not impossible? Why should Why do you want us to keep lying? Right? It's sort of a pushback on Parnass to go, hey, you know how we've been vaking it all this time? Can we just admit we've been vaging it exactly, and that's why the agile movement then got suppressed. Actually, and that huge message has been pushed into all kinds of frameworks and things that go back to the goal
of giving us certainty and structure and order. Yeah, which is I think against the spirit of that original manifesto. I'm not sure how many people behind the manifesto actually realized that they were they were taking a swing at something really, really big when they said that really big that actually swung back quite successfully. I remember the we talked about this in one of the Fusion Geek outs. The Ider project, the giantic project was the whole thing was get the
first billion dollars spen because then they can't cancel it. So we can't tell is what we lie until they're too committed. Then we just work on what needs to be done. Yeah. Yeah, And a number of mega projects that are like that, where it's like, if you actually knew the full scope of this, you may not do it, and we really want to do it and it will be beneficial, but we can't tell you the truth because then you won't do it. Yeah, that's a great note there,
Berry. This is so cool. I mean You've really made my day and I'm loving this well, and congratulations on what is clearly a very cool book which you can get for free, but do yourself a favor at least pay the minimum price for you absolutely lean pub you have a choice. Yeah, and it looks like a fun read. Yeah. The artwork has been done by my eleven year old who's incredibly proud of it. Right, I think that's one of the coolest things. And it's our project. But what's their
name, let's call them out. Alexander all right, who did the artwork and is now demanding royalties and I like it even more. Yeah, Well, this is great and you've changed the world. So thank you for spending the time with us today. And well, I'm sure we're going to talk to you again. Cheers, guys, see you soon at a conference somewhere, Okay, cheers, and we'll talk to you next time on dot net
rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two thousand and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code. See you next time. You got Jamtavans and