Immutable Architectures with Michael Perry - podcast episode cover

Immutable Architectures with Michael Perry

May 18, 202353 min
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

What's an immutable architecture, and why do you want one? Carl and Richard talk to Michael Perry about his book The Art of Immutable Architecture and the power of historical models. Michael talks about different designs for immutability, the ability to always look back through data, to avoid conflict between resources, and the advantages of eventual consistency. As Michael says, you already use immutable architecture - look at Git and how you only add new files to the system, always able to get back to a previous state! The conversation dives into implementing architecture in a way that helps to show where immutability makes sense.

Transcript

How'd you like to listen to dot net rocks with no ads? Easy? Become a patron For just five dollars a month you get access to a private RSS feed where all the shows have no ADS. Twenty dollars a month will get you that and a special dot net Rocks patron mug. Sign up now at Patreon dot dot net rocks dot com. Hey Carlin Richard here. As you may have heard, NDC is back offering their incredible in person conferences around the world, and we'd like to tell you about them. NDC Oslow will

be made twenty first through the twenty fifth. Go to NDC Oslo dot com to register. NDC Copenhagen is happening August twenty seventh through the thirty first. The early bird discount for NDC Copenhagen ends June second. Go to NDC Copenhagen dot com for more information. NDC Porto is happening October sixteenth through the twentieth. The early bird discount for dc Porto ends July twenty first. Go to EDDC Porto dot column to register and check out the full lineup of conferences at

NDC conferences dot com. Http timeouts Database deadlocks in software, it's not a matter of if things fail, it's a matter of when one mishap like this and some valuable data is lost forever. And these failures occur all the time, but it doesn't have to be this way. Introducing end service bus, the ultimate tool to build robust and reliable systems that can handle failures gracefully,

maintain high availability, and scale to meet growing demand. For more than fifteen years, end service bus has been trusted to run mission critical systems that must not go down or lose any data ever. And now you can try it for yourself. End service bus integrates seamlessly with your dot net applications and can be hosted on premises or in the cloud. Say goodbye to loss data and system failures, and hello to a better, more reliable way of building distributed

systems. Try end service bus today by heading over to go dot particular, dot net slash dot net rocks and start building better systems with asynchronous messaging using end service bus. Hey, guess what it's dot net and rocks. I'm Carl Franklin and I'm Richard Campbell. We're getting all of these shows done so that we can go travel for a bit going on adventures, going on adventures. We'll do more shows while we're adventuring too, And we're gonna be at

Techarama in Belgium. And I believe you're going to Dot Net Developer Days twenty twenty three in Poland, right, yeah, in Warsaw Warsaw. I'll be there, but I think I have the opening keynote. Actually, yeah, that's cool. I don't know for sure, but it's possible I be there, But if you want to check it out and you're anywhere near Poland in October, it's net dot Developer Days dot PL. It's a it's a great conference. I went there a couple of years ago and I had a great

time. Yeah, we had a ton of fun there. Yeah. Okay, anyway, let's get right to the meat of it with better no framework. All right, man, what do you got? All right? So this is a blog post by Eric Elliott March thirty first on medium dot com. This guy has written a pseudocode programming language. I almost said pseudacu.

A pseudo code programming language for large language models. And it's called pseudo lang sud O l A n G. See what he did there, Yeah, and so he's been using pseudocode to express ideas to large language models since GPT three. However, up until GPD four it didn't work extremely well. But then he's working on it, and I haven't used it. But there's a whole bunch of statistics and stuff in there, and he says, here's some commands and some modifiers and examples and it, and it looks good. I

don't know, it seems pretty complex. I mean, isn't English a pseudo language in the first place? You would thinks so, yes, But I guess it's somewhere between a programming language and natural language. But there I thought the whole idea of large language models was to use language, right, I think that's what you're saying, right. I don't know. I can't really judge it because I haven't really read through it. So I thought it was

interesting and came across my desk. And since you know, we're in the hype cycle of GPT large language models and all of that stuff, people are going to start dipping their toes in tools to make them more powerful, right, So somebody should check that out and get back to us and let us know how cool it is. All it's all explorations at this point, but interesting ones. It is certainly interesting. It was talking to us today Richard

grabbed a comment offer. Show eighteen o nine is one we did about microservices architecture with our friend Shawn Wildermouth. Step back in follow twenty twenty two, Bart Lenois said, Carl, Richard and Sean thanks a lot for the show. These days, you seem to be doing something wrong if you're not using microservices and Kubernetes. It's like everyone forgot about the kiss principle and they have

this battle with almost every client. There's nothing wrong with a well architect monolith or service already an architecture and deploying a handful of services on Azure, app service or functions as minutes of work with minimal skill required. Only a small percentage come to the point where they need full blown microservices architecture, and if you're googling for it, you're not one of them. Listen, Bart,

here's your solution. When the customer says I would like some Kubernetes please, you get us a bottle, put a big logo on it says Kubernetes, and you squirted on them. And then if you're really smart, you say no. And I got to remind everybody that the kiss principle is and I quote you gotta lose your mind in Detroit Rock City. But I really appreciate this comment from Bart just because it's like, listen, there's not one way

to solve every problem. And if you're determining a technology before you've argetted an application, you have are on the wrong path. Right like that, It's that's just a given. Let's look at the problem space first, then we can talk about what we build. So Bort, thank you so much for your comment, and a copy of music cobuy is on its way to you. And if you'd like a copy of music co buy, write a comment

on the website at dot net rocks dot com or on Facebook. We publish every show there and if I read your comment, I'll send you a copy of music cobuy. And you know, you can follow us on Twitter. That's all cool, but it would be really cool if you followed us on mastadon because that's where all the cool kids are these days. I'm at Carl Franklin at tech hub dot social, and I'm Rich Campbell at mast social and

send us to two we'd like to hear from you. And that brings us to our friend Michael Perry, who I was shocked that we haven't had him on dot net rocks, but we did have him on the tablet show back in the day. So I guess everything just ran together in my brain and I'm well, it's it's not like we don't see mcconferences all the time, all four times. He's part of the circle. But we were roommates. Embarrassingly, we never did this show for crying out loud an MVP summit.

Weren't we weren't we roommates? Yes, we were. We were roommates. Yeah, we didn't choose to be. It was just like, hey, I know you, so let me formally introduce Michael Perry. Software is math. Every class is a theorem, the compiler is the proof, and unit tests check our work. Michael Perry has built upon the works of mathematicians like Mark Shapiro, Pat helland and Leslie Lamport to develop a mathematical system for software

development. He has captured this system in a set of open source projects Jinga and assisticant. I love assistant. Yeah that was my daughter's name. What yeah, Yeah, Well, when my daughter Kayla, whenever she thought her name was assisticant. No, but whenever she would help her mother out at a at a show, then you know she would ask, Hey, can it come along and be your assisticant. Oh that's so cool, that's great. I love that. It's almost like something that goes in a squirt bottle

for Kubernetes. Isn't it a little assisticant? And yeah? Well anyway, great, great to have you on the show. And just your bio sort of sets the tone for what we're going to talk about here, doesn't it. This is really fascinating to me. Yeah. In fact, the other open spurs project is pronounced in Janaka. It's oh what did I say? Jenga yeah? Jinga yeah, yeah. Yeah. It's not the the tower of bricks that are going to fall down on you. It's it's much more

staple than that, ji Naga. So what is Janaga? Yeah? Janaga is a platform for UM for managing data in a in a distributed system UM. One of the great examples is in a mobile application. If you think about each mobile device being its own separate node within the within the network UM, then you can you can have your your data on your mobile device, interact with it as a user, and then it uh um, it kind of sinks up with the replicator, uh and you get that eventual consistency.

So yeah, Janaga is a way to to turn your micro services mobile applications, web applications into these uh, these small isolated nodes that eventually will reach agreement. It's interesting. And Assisticant is what. Yeah. Assisticant is a is an MVVM framework um that is based on the idea of dependency tracking. If you remember, you know, Steve Sanders Sanderson's old knockout, Yeah your

library. Yeah, that's uh, that's the same the same algorithm that it It figures out what depends about what, so that when something changes it it notifies the dependent Wow. And so you're talking large object models, then yeah, I tend to use assisticant for the the smaller focused like this is going to be a view model for just a single page, or even a composite view model with the you know, you know, a page that has a list in each one of those elements in the list is its own smaller view

model. That's what I tend to use assisticant for, not so much for the larger ones. So some m MVVM frameworks come with sort of communication tools of one kind or another, don't they. Yeah, yeah, yeah. A lot of times when you see MBVM, it's like, how do I implement I not if I reputy changed, And I'm like, oh, let's take a step back. Let's not try to implement the programming problem at hand,

but instead try to specify the desired behavior. Yeah. Sure, and then let the computer figure out how to how to give you that that desired behavior. Or we could just use Blazer It's true. Yeah, yeah, Blazer is a fantastic isn't it great? Just love it? All right, So we're talking about how does this incredible bio here that you know that you've built these this mathematical system for software development and I guess your two projects that

you talk about illustrate this. But um, how does that align with the idea of immutable architectures? Yeah? Yeah, So this is this kind of goes back to UM. If you look at those mathematicitions I mentioned their Mark Shapiro, that first one he and UH and his co author Nuna Pergucia basically published the very first c RDT specification. That's a conflict free replicated data type UM. This is a mathematical object that has some amazing properties in that it

will reach eventual consistency. So it will reach consistency as soon as all of the nodes have received all of the information eventual consistency. Because this doesn't come up often on dot net rocks, but this is the idea that I have object A with a state, and when that state changes, eventually Object B that is some sort of replicant of it is going to change, whether that's a database or whatever, eventually. Yeah, yeah, exactly. Yeah.

And and what's really exciting about that so that you might have heard of algorithms like paxos that are based on consensus. So so these different nodes have to collaborate with one another, They have to you know, chat back and forth in order to come to an agreement who's going to have the correct state. In the case of a c RTT, the way that the data structure works will guarantee that as soon as they all receive the same information, they are

in the same state. They don't have to go through this extra this extra checks and balances. Yeah, it's pretty cool stuff. That is cool. Yeah, Well, it's interesting to have the stuff's sort of solved models, right, Like they're exemplars or design patterns even of how you should build these kinds of problems, like you should not invent a new implementation of eventual consistency. Right. Yeah, it's very easy to be wrong. It's very difficult

to detect that you're wrong and it is a solved problem. Yeah, yeah, it's UM. It's very similar to what I was just saying about, write the specification and let the computer figure out the implementation. Computers are really good at computation. Yea. They can take an equation, they can solve it, they can give you the correct answer. You give a human that

same problem, you're not going to make mistakes. UM. But now if if the human approaches that problem but instead and writes a specification, now what they're doing is they're they're giving the computer the equation. They're saying, I want you to produce the system that does this. Um. You know, like you know, taking it back to cr dts UM. One of the one of the first c r dts that UM that the Mark Shapiro and No Prusia published was tree doc. This is a cr DT that UM that supports

collaborative editing of text documents. So, um, so, if you're familiar with Google Docs, Google docs doesn't actually use this algorithm, using something else called operational transforms, which are known to have defects. But if they had used tree doc then basically by specifying okay, these are the rules of the r DT computer, you come up with the implementation. Then by using the mathematics of crdts, it would be able to come up with a correct implementation

of collaborative text editor. Michael, did your interest in computers and maths sort of grow at the same rate or what? Was there a period in your life when you were all math and computers came out to that man, Yeah, I think I think they pretty much did grow at the same rate. That's cool, but yeahs, I've just always loved the mathematical side of software. Yeah, that we've seen it as applied math. That's a very unique place to come from for software developers, isn't it, because you know,

you get to experiment with both models at the same time. Yeah. Yeah, and you end up with some really surprising results too. You know, some things that that people don't expect to work, Like when I show people assisticant um it's like, well, yeah, it's it's a very simple idea. You write it um like a property getter, for example, write a

property getter UM. And if it's going to reach out and get values from other uh from other properties, then you know that it depends upon those So why not just have the computer watch while it's executing that and see what it does. And now you've just tracked your dependencies and so whenever those things change, this property has changed, um. And so it's like, well, yeah, that that makes sense. That follows logically one from from the next.

You've basically just told me a proof uh that that you're going to get the correct behavior out of a dependency tracking system. And then you actually, well then here's an implementation of it. Let's run it and it works, and it's like it's magic. It's like I just explain it to you. Very reliable math at that. Yeah. Of course, many of the original computer people were mathematicians. I mean, fundamentally we're still all using von Yuman

machines. And von Neuman was a mathematician. The question is if he wasn't born in a day where there was no computers, would it be an a computer scientist rather than a mathematician, Like you didn't. You didn't really have a choice but to be a mathematician and end up in computing. That was the path for that time, that era. Yeah, yeah, that's true.

And yeah, if i'd if i'd have been born, you know, you know, fifty years earlier, Yeah, I would have just been a pure mathematicians, you probably would have gone into maths, right, Like, that's kind of you don't really have a choice. It's like, if you like that purity of problem solving, that's pretty much the only place it lives. You know. When I was in college and taking a computer science course, the computer teacher said you kind of gotta love math to get into it.

And she couldn't have been more wrong. I mean, the right right brain programming ideas were blooming in the nineties, like and she was just out of punch. You know. It wasn't all Fortran and you know and calculus programming and stuff. It was you know, we were talking about business logic, and we were talking about language and all of these things that computers could do. So so I got into it from a rape brain. Obviously, music is like, you know, music and computers sort of grew in my

brain at the same time. And and you know, so it's fascinating to me to find a species from the other side of the world the left brain approach, um, because you know that that's completely opposite to my experience. So, and I don't know about the rest of the you know, people out here, but the hardcore engineers that right, operating systems and stuff and right, you know, serious stuff with computer science degrees, they're they're certainly

more into the maths. But guys like you obviously who thought about really carefully thought about how these mathematical mathematical models play out and software. Yeah, isn't this a wonderful industry, so much diversity. There's room for all types of thought. Absolutely, Yeah, and they they dressed different parts of the problem space. Like it's not like us is running out of work. There's lots

and lots to do. But yeah, the the idea that you, you know, you have to love math to be involved in computing is just no longer correct. Yeah, there are elements of computing that certainly well served by math. But look, there are good tree negotiation strategies. They're APIs now you don't need to write one. Oh, yeah, you're good I still think that in order to do just basic stuff, especially any kind of UI

kind of need to know a few a few things. He needs to know ratios, he needs to know, you know, algebra a little bit, but it's not it's about it yet, you know, maybe a little trigonometry. I was working on some connects stuff and I needed some trigg and that was mind blowing. Yeah, and I think that I think there's a difference between them. The subjects of math that we just talked about, the trigonometry and algebra and even some calculus. Um, but then there's also the mathematical

thinking, this logical way of thinking. So um to one thing, I like to say when people ask, well, do you have to be good at math in order to program? And no, No, math and programming they're completely different to operations. In math, you're taking symbols and you're you're manipulating these abstract concepts in order to come up with a solution, whereas in programming you're taking symbols and manipulating these abstract concepts in order to come up with

a solution. Comparing the different things, yeah, just the shape of the symbols. Well, and really, who's the compiler today exactly. That's the difference. Yeah. So I like to think, um, it's not that you have to be good at math in order to be a good programmers that if you're a good programmer, you're good at math. You just didn't realize it, right, Yeah. Yeah. If you learn the if your learn the rules of symbolic math, you'll find you're already thinking that way, right.

Yeah. So where's the immutable part come from? Too? Yeah? So, um so it turns out that's one of the um one of the best cr dts. Um is a directed a cyclic graph of immutable records. So one more time, CRDT, just one more time. The Yeah yeah, CRT is a conflict free replicated data type. Okay. Um So, so by a data type we mean a data structure plus operations. Um So,

just like a linklest the data structure, if you will. But it's also got some operations and so that pair, that combination makes it a data type. That's ok, it's useful. Um. So, a conflict free replicated data type is a data type some structure and some operations that live on different replicas, different different nodes. Um. And it's conflict free because as you perform those operations, you will never get this, uh, this cr DT to be in a state of conflict on two different nodes. In other

words, no dependencies. It's it's it's it's pretty cool. Um. We can actually tie this right back to the best implementation of an immutable cr DT that just about everybody listening to the show should be familiar with, and that is GET. Okay, In GET, you you make a change, you you create a commit. A commit is an immutable record of the change that you just made, right, and part of that record is who made it

when, what was the commit comment? You know, where the contents of the commit, but also what is the identity of the prior commit um? And then GET will take all that together, put that into a data structure, compute the SHAW to BT six hash of it, and say, okay, that's the identity of this commit. Right now, you create another one, okay, collects all the information, and then what is the identity of

the previous one? What's the previous shot? I got it? And so by creating that that reference from one record back to the previous one, you're creating a directed a cyclic graph. So it's directed, it's always pointing back to the past. Here's the prior commit, here's the prior commit. UM. And it's a cyclic because there's no way for you to make a commit that will then refer to something that comes loops back around and points back to

itself. Yeah, so sort of the journal of pattern. Yeah there if you will, Yeah, yeah, exactly, yeah and and um so so I would I would call GET an example of an immutable architecture UM. There are there are a few other examples of immutable architectures UM, and so some are very similar to GET, like blockchain UM, in that you elect a bunch of transactions, you have the hash of the previous block, and you compute it's hash, and now that's the reference to the new one. Right.

The difference is that blockchain requires proof of work in order to make sure that that chain never never forks, whereas GET forks are a feature, not a bug. Yeah, we're calling branches and that's how we do work. But the main thing area is no information is ever destroyed. Right, it's immutable, it's always there, exactly. I mean we've advocated for this and

databases for the longest time. Is these sort of insert only tables journal style tables because we want like a bank, we want a record of every motion of money, every change, even if the sums come later. Right, and more important is just the motion. Yeah, exactly, exactly, Yeah, you need that immutable history. Yeah, except then when you got to do a report. So immutable architecture is one where data is never destroyed, right, right, Not great for reporting, Nope, not great for reporting

at all. Yeah, use something up for that. Yeah, but but yeah. Another another example that that people might already be familiar with is event sourcing. So that is a third immutable architecture. I would say in that list that basically, UM, there has to be some stream that puts those events into a sequence, or at least in a piece wise sequence. So for this particular aggregate, these are the sequence of events that have occurred. And now from that you can compute the current state. Right, And so

a lot of people build applications using using events sourcing UM. In in my book The Art of Immutable Architecture, I introduce a fourth immutable architecture that I call historical modeling that is also suitable for building applications, but has some of the advantages of get UM as compared to things like events sourcing. Okay, so your product Jannaga, did I say it right? Yes? Okay,

perfect? So you said this is sort of it kind of sounds like an actor model pattern for mobile, but not really because an actor model has like these these actors um in memory on a server somewhere that represent a client. Yeah, isn't that right? So this is a little bit different, isn't that? Yeah, it is a bit different. Um. And And another significant difference is that with the actor model UM, the actors have mutable states.

Right, you send a message to an actor, it can load a state, mutated, save it back and tell you what the answer is, right, whereas with an immutable architecture, mutation forbidden, right right, just not not a thing you must love f sharp I actually do. Yeah, I don't. I can't do to program it very much. But you know now that that dot net has records, Yeah, it's like, oh man, I'm all over those records. So so yeah, the the new version of Jenaga that I'm working on right now, Jenaga, that net um is

taking full advantage of those immutable records. Yeah, yeah, yeah, yeah. The wait till wait till you see how the how the code looks. It's amazing that's very cool. Does the enthusiasm for this? I mean, to me is a guy who's usually worked on performance problems. What I like about all these immutable architectures is they perform very well because they avoid contention, which is usually your performance problem. I mean, is that a reason to

do it? Like? Why else would you want immutable architectures? Yeah? Contention, that's um, that's really close to that idea of conflict free right. UM. So so yeah, if you if you have two nodes that that need to kind of battle it out in order to figure out what the current state is, so you've got that Paxos kind of thing happening, then then you're going to waste a bunch of cycles in order to UM to resolve

those those conflicts. Yea, but but yeah, if you UM, if you have a data structure that that proves that you will not have conflicts even if you are on just a mobile phone and you're disconnected from the from the network for you know, any period of time when you finally do reconnect, there will be no conflicts. Then you don't have to waste time resolve in those conflicts. And UM, while you're disconnected, the user is interacting directly

with that device. They're not waiting on any you know, connection to the network. So UM. So yeah, it gives you some some performance benefits UM you know as well as that that scalability. Is this product and open source product? Does it work on all dot net platforms? What's the story there? Yeah, this is this is an open source project. Um UH. It is is currently uh primarily in JavaScript. UM Jenaga dot net is the the newest incantation of it UM but Jaga JS is UM is for for

the client and for the server. So there's uh UM there's a UM you know MPM installed Jenaga and there's a human installed Janaga dash server. UM. So when you when you running on the server, UM that's to be run

as an express uh node application and it gives you some some endpoints. Then that give you access to its um its internal replicator and then you install Jenaga as as a client in your React application or React native and um UH and then that connects to that replicator and UM and genagages on the client can also UM use UM use the web dB or index dB for um UH for the web in order to support progressive web aps, so you can go offline with your PWA. I was, I was very I did some work with the

nextb Uh last year and I was really impressed, you know. But I was using a wrapper for Blazer around it, which turned out to be great, because man, that's scary stuff. If you're going to deal with it in JavaScript, more power to you, my friend. That's yeah, it's so scary code. Yeah, yeah, Well I'm aiming toward that, uh, that place where as a developer you just have to write the specification, yeah, and then let them let the language, let the platform figure out

the h the behavior. So. Um. So when you're doing a lot of disconnect to mobile apps, you're thinking about, oh, I'm in you know, I'm in connected mode. I'm in disconnected mode. I have code that writes to the local database. I have code that talks to the API. You know, I have code that that synchronizes things together. And so you're you're doing a lot of that bookkeeping yourself as a programmer, right.

Um. But but in Janaga, when I'm when I'm going for is a place where you can say, yeah, here's the fact the user just um just did this thing. You know, they entered some stuff into a form impress safe award that fact just just a jobascript object. Yeah. And then um, here's a specification which says, um, you look for facts of this particular shape where these other related facts exist or do not exist. Yeah, so you write this this pretty much this mathematical specification for you know what

it is you're looking for, and then give me a projection. Turn that into a thing that I could just bind into the screen. Yeah, and then let the run time figure out how to save that to local store, how to talk to the server, how to synchronize those those data sets. Awesome, And that's pretty cool. And I'm gonna interrupt for one moment for this very important message, and we're back. It's done at rocks A Richard Campbell. That's Carl Franklin. Hey, hey, hey, talking to our

friend Michael Perry, late of the Tablet show, very late. I think we did the last tablet show in twenty fourteen. Website doesn't even come up anymore. And the Melvin the website has come up. Who knows that maybe we can at the moment, Yeah, turned his statical. It's always archive dot org. Maybe I'll pull it, pull it up that way, and and it's still apparently on on where all the podcasts are. So, yeah, the podcast engines are still got it. Um. But we're talking a

bit about immutable architectures and I sort of get this. Where does CQRS come into play? And I guess I'll define the acronym command query responsibility segregation because now that you know the words, it makes way more sense command query responsibility segregation, I can. I can boil it down and tell me how. And this is purely from memory. Basically, you have one data source for quarrying and another data source for writing. Is that it? That's one way

of implementing it. Yeah, yeah, it's not necessarily that, um okay as um yeah, Oudi Dahan and and Greg Young would would point out not necessarily yeah, sure, but yeah, that's that's one that's general. Just that I learned that when you write to a data a data source that has eventual consists stancy to the one to the second data base or data source that we acquery from. Right, Yeah, that that is the most common way

of doing things, and it makes a lot of sense. Like you were just saying, you wouldn't want to run reports off of an immutable history of records. Um, that's not what that's for um, so use something else for that. And that's perpectly what cr QRS means to me is that, um, you're going to have something that's really good for those those reports. For that read side, that's your projections. Yeah, so run your reads from your projections, um, and then your rights. You're sending those into

some transactional system. And if that transactional system happens to be a directed a cyclograph of them middle records, you're golden. Yeah, I hear you got events sourcing on your events, So I got this to a subliminal events sourcing every other words in chewing events starcing. But I do appreciate the idea of

optimize the ingress path and optimize the outgress path separately. Right right, that there's an ingress path for orders to be entered, and then there is that One of the output paths is fulfillment, but another one it can be is reporting. They don't have to be Why do one thing everything in one place and basically have suboptimal versions of all of it? Right right? Exactly? Yes? Yes, And now imagine that you could just write a specification what

does your reporting database look like? Yeah? And then computer you figure out. Okay, whenever this event happens, go update that table. Right, that's computation, that's machines. Yea, you could solve that. But I mean this is where you get into that sort of relationship mapping thing. It says, this is what the incoming stream looks like. Maybe it's blobs, you know, it doesn't really matter, whatever it comes to, and it

maps it out asynchronously the transaction. It has already been received, but a syncreitly now distribute it to various sources that care for it. Yeah. Yeah, oh and you've got to have a queue in there somewhere, or at least at least one. Yeah, that's okay. Wow, that that is UM. So I want to kind of kind of kind of tie this back to UM what we were just talking about with event sourcing. UM I do I do uh uh I do offer an alternative to event sourcing as a different

immutable architecture that I call historical modeling UM. Whereas event sourcing is this, UM is this at least piece wise uh linear set of of events a historical model UM. They they don't have to be linear, they don't have to have occurred in a in a specific order. Um, it's not tied to the data per se, it's tied to them. Yeah yeah. Let me let me see if I can give a practical example. So, um, so yeah, you you placed an order, say yeah, you know as

a customer. Um, and then what's the next thing that's probably going to happen. Well, the you know, the shipping department is going to pick that up and they're going to create a picklist. But also the accounting department's going to pick that up and they're going to create an invoice. Yep. So the picklist and the invoice are both historical facts. There there happened,

They happened. There's their decisions that somebody made based on that order. So if you model that as a tree, you'll see here's the order at the top, and then the picklist and the invoice are two leaves of that tree, both pointing up to that common route. So it makes a little triangle. Yeah. Yeah, I would argue the picklist is fulfillment. Shipping is another task. Again. You know that you end up with this series of

artifacts related to the order, and they're all done to different people. Sometimes they're sequential. You need to be picked before you can ship and in theory, you shouldn't invoice until you ship. Yeah, but the tree architecture sounds like handles that. However, you want to branch it. Yeah, yeah, exactly. Yeah, so shipment is another historical fact that that points back to the pick list. The shipment doesn't care about the invoice, no,

so it's just pointing up to the pick list. Yeah, there's a there's an accounts receivable, which is actually just a report that you run from invoices. So their accounts receivable itself isn't not a new factor, but the payment is another another element is another fact. Yeah, and that points to the invoice, but the payment doesn't point to the shipment or to the pick list. So you've got this this this tree, this graph that branches m and

now do now do a partial return or a warranty replacement? Like and you can tell the architects in the room because you get all really excited, good white boards drawing for you. Yeah, but you're looking for those dependencies and what they're related to and where we can make them execute independently like this. You're also unique units of work for development too. Yeah. Yeah, this is how it's always that question of how do I get six people to work

on this. It's like, well, because you peel out the one piece, the payment engine, and that goes to one person, maybe two. If you're doing a little pair of programming, you want two sets of eyes on things, and then others are on other pieces. Like you have these, you know, areas of concern that have discrete interfaces on them that allow you to paralyze development, right, exactly. Yeah, And so the same reason that I shipped two copies of Mythical Man month to every manager so they

could read it twice as fast. And I'm going to read Michael's mind here. He's saying what I would do is I would write the engine and then have one guy right the spec for everything, and then the engine does it all right, yes, yes, and that's what we want. Yeah. And the person who writes the engine doesn't even know about your problem, doesn't care, and he's long gone No, he's got a problem factory somewhere. Exactly. Oh, one of my favorite jokes. All right, these are

a whole bunch of insider architect jokes. I'm pretty sure we're making fun of ourselves. I had a problem, so I use Java. Now I have a problem factory. It's an old joke, but a good one, and you have to say it's slower than that because you know Joba. Yeah right, all right, I'm not shouldn't make fun of Jab. Those guys do good work and they're more like us than different. That's true. It's not fair at all, and there no reason to be mean to them. They

have oracle for that. Man. How much time do you spend doing architecture that, Michael, I mean you work for an agile practice group, right, yeah, yeah, I work for Improving and yeah, so we we offer consulting agile training, and so my role at Improving is to help help other improvers grow their careers and also, by the way, bill clients.

So so yeah, I'm um, yeah, I'm often running a team at a client, So in addition to helping to to grow that team, I'm in charge of the architecture for the project as well as often doing you know them you know, technical lead role, right because I mean presenting a directed bicyclic graph to the average developer, they're they're like, where do I log in? Like, it's it's not a product, right, it's a thinking model. And then you get into questions about well, how do I implement

this? Like, is there a library that will drive me towards a Dagum? I don't start people there. Um okay, yeah, so I start people first with analysis. Let's use a director bicyclic graph in order to analyze the problem, and that uncovers all sorts of edge cases, but then also gives you a way to document them. So you say, okay, you

know, like the the the returns that you were just talking about. Yeah, So the breakdown and giggles we just had was actually working through the graph of a retail workflow or an e commerce workflow, right, I mean I've done so many times I don't even laugh about it anymore. Right, this is just like it's the number of times you figured out, are we going to do back ordering? How do we do returns? You know, do

we do unify where you can deliver back to the store? Like yeah, and and just drawing out those diagrams, like I think you forget how weird it is. Yeah, yeah to think about business processes that way. Yeah, And a lot of times people will draw those diagrams as as flu charts or as state transition diagrams or you know, the more modern equivalent to draw those ash as a sequence of events. You know, do some events storming,

right. Um. But the the the diagram that I that I propose is one where you've got an ellipse that represents one historical fact, one decision, right, and then it's got arrows pointing up to the predecessors, those decisions that had to have come before, right. Um. And so now you can walk this tree or this graph really in um, in any order as long as it's from the from the past to the future, from the top down. Um. So you have to follow those arrows backwards, I

like from arrows backwards. But uh, but but yeah you can. You can. You can take any of those paths at any time, and you can describe all sorts of different scenarios with that same graph. So it ends up being a lot cleaner, a lot easier to understand than a flow chart or a or state machine that does the same thing. Yeah, I do appreciate that it's more representative of the problem space ultimately to say, it's sort

of really organizing by artifact, Yeah, exactly. And so then once once I've got people thinking in terms of a director day cyclic graph to analyze the problem. Then we can start talking about how do you implement based on right, And even for that, I don't jump to hey, let's use Janaga Instead, I say, all right, you're used to relational databases. Relational databases have foreign keys. A foreign key is a reference to another record.

So what if every record was an immutable historical fact and those those foreign keys were the coiners to your predecessors. Let's actually build a database like that. I've I've done a number of systems that way, and it's it's really really powerful. What about structures and data like like I'm thinking of lists of objects

of things. If you already use immutable disimmutable architecture where you have records, now you've got duplicate records that you know, no, this is the current one, and these are the historical ones that could get Harry, couldn't it. Oh yeah, that's that's that's a really fun problem. Um. So yeah, there's a pattern that I that I go through in the book called the mutable property pattern. Wait, you just said immutable. You called this

the immutable immutable property. What's he broke his own rules? Yeah, so this is a way all the other farmers on tattooing think of you because yeah, this isn't right, but the um but yeah, the idea is that you were going to simulate mutability using immutable records and um. And so the way that I that I simulate a mutable property is that I've got a historical fact that says the value this property is this value, and the prior value, the one that I just replaced, is that one. So it's the

predecessor. The pointer to the other fact is the previous version of that of that property. Now, what's really cool about this is if you think about this not in terms of a sequence, but in terms of a graph, you can have concurrent changes to a property. You're can have somebody and you can have on their cell phone somebody else on the website, and you both create these facts that point back to the same prior version. Right, and so you've created a tree. You're okay with a tree. Then that two

facts could be pointing back to the same historical facts. Yes, yes, exactly, you are crazy, mister Perry. But but yeah you're used to that as well. In get I'm just curious. No, all right, So this is how strings work, isn't it inherently how strings are immutable objects that when you say, you know, my string variable equals this and then equals this plus that, you're actually creating a new string that from this plus that, and the string that just had this in it is gone. It's

it's marked for death. That's it's unusable memory. Now freed memory. Yeah, so it's well, it's not immediately freed, it will be it is. It is immutable, so you've you've got that part. The thing that strings don't do is keep track of what was the previous string in this variable, right, Yeah, you can't undo a string for example. Yeah, but but string dot undo. Yeah, taking it back to to get It's perfectly fine to have two different commits that point back to the previous commit.

You've just cut a branch. Well, yeah, all perfectly fine until you you try and merge and nobody's fine. Or you try to do a link query to to find some records in the in your record list. Uh huh, yeah, okay, let's let's put a pin in the link query one. Okay, yes, that's really cool. Um, but but merge, I do want to I would do want to talk about that. So in in critt is one of the One of those operations that this data structure needs to perform is a merge operation, but it's not the same as a get

merge. Get is a cr DT. So if you were to if you were to fetch from a remote, now you're performing a merge. You're taking information in that remote and you're merging that information into your own directed a cyclic graph on your repository. Actually performing a get merge creates a new commit, So that is an actual change, that's an update, that's creating a new historical record. And now that you've created that, you can push, which

is a merge in the opposite direction. So so yeah, the thing that we call merge and get, which is so painful, it's actually not a merge. And the thing that we call emerge conflict in get is not a conflict in the term of in the CRDT world, because as a CRDT it

should be conflict free, right right. What's actually happening is the interpretation of what this history means now has has been called into question because you've got concurrent edits, And that's exactly what's happening when you've got these mutable properties that two people change at the same time. Now you're calling into question what does that mean? And the data structure itself, it's not in conflict. It's got

a graph that has two leaves on it. That's perfectly fine. And now your interpretation of it is where you say, oh, was it this phone number or that phone number? Which one is it? And then the way I like to do that is to allow a human, just like can Get, to resolve that conflict, to make the decision which is the correct phone number, and then create a brand new historical fact that points both of them, just like can Get when you do emerge commit it points to both of

its parents. So it was looking at Get for you. The sort of impetus of this pattern that you're describing. I mean, you seem to be referring to Get a lot or was it like, oh, Get does this thing that's in my brain? How fortuitous? Yeah, it was. It was a little bit more of the latter, because I first had this idea of um, of you kind of paying attention to the history rather than the current state around two thousand and one, and I wasn't um yeah, I

wasn't familiar with it. If it was around at that time, Um No, I don't think so. Yeah, but but yeah, then then I kind of started to evolve this and then I saw, oh, there's event sourcing. Oh, that's that's also paying attention to the history and not the current state, but in a different way. Interesting that that feels about. Right, Oh, here's a get Oh that's doing this thing in a much more familiar way, but just for source code, whereas I want to do

this for every vacations in general. Yeah. Oh, that's so a lot of things. It all kind of felt familiar, well all happening at the same time. All right, well that's wow, that's really cool. So I've just one more question before we go. When you go out on a clear night and you look up at the sky, do you see two full moons? Uncle? And this are too? You know it's got a bad

motivator. No, seriously, this is mind blowing stuff. And just to be able to think this way and apply it to common things that you're using everyday life and now could it how could it make things better? Yeah? I love that. Yeah, it's it is. It is amazing when when I'm able to bring a development team along with me and we use this as the way of thinking about a product about the way that we developed the product.

It really comes up with something, you know, much better than we could have done that at it. What's the name of the book again? The book is called The Art of Immutable Architecture Excellent and you can get it wherever fined books are sold. I take it. Yes, you can go to immutable Architecture dot com. That will give you a link directly to it. Do you have an audible version? Um, I do not yet have

an audible version, but yeah, it would be a great idea. Well, don't ask me to read it, because I love all nobody expects all the w T apps. I will read it however. So yeah, this is great stuff. Michael. Thanks, thanks very much for being with us today. Thanks, thanks, thank you. All right, We'll see you

next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at dt n et r o c ks dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two thousand and two. And make sure you check out our

sponsors. They keep us in business. Now go write some code. See you next time you got a band,

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android