How'd you like to listen to dot net rocks with no ads?
Easy?
Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month, we'll get you that and a special dot net Rocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Happy Chris Mahana Kwanzadon. It's dot net Rocks. I'm Carl Franklin, an.
Amateur Campbell and we timeshift. So this is actually being published in January.
That's right, January ninth. So we hope you had a happy holiday anyway.
Yeah, I'm sure it was great.
Yeah, we haven't had ours yet as of this recording, No December nineteenth, but you know it's going to be a good show. Our old friend Thomas Betts is with us. And before I get into better know framework, I just want to check in with you, Richard Campbell and see how your holiday is shaping up.
Oh, now we're gonna we're shortly after this, like this weekend, we'll go to the city for the week Lots of friends to visit a few different parties, go to spend time with the girls. It's probably been the leak out by now. The youngest is pregnant, so she's like, hey, can I bourrow one of your old winter coats because mine don't fit anymore.
Yeah. Kelly's daughter had a daughter and she turned one last week. Oh yeah, right, so we had, you know, the obligatory smash the cupcake in your face birthday party.
That's the thing.
It was pretty awesome. Yeah, all right, well, we've got a lot of stuff to cover in better know framework. It's history day here, so roll the music, all man, what do you got? So? I hadn't done this in a while, but you know, nineteen thirty two, which is our episode number, was such a jam packed year that I want to talk about some things that happened. And Thomas, I know you're there if you have anything you can jump in.
You two.
Richard, of course you are the history guy. So January fourth, British colonials in India arrested Mahatma Gandhi and put him in prison. January twelfth, US elected their first female senator, Hattie Wyatt Caraway, who represented Arkansas. February seventh, NASA astronaut Alfred M. Warden was born in Jackson, Michigan. He went on the Apollo fifteen mission, which saw the use the first use of a lunar roving vehicle, a little Buggy, Little Buggy.
March first.
Lindberg plays a big role In nineteen thirty two. The Lindberg kidnapping occurred, where the twenty month old son of famous aviator t Arleson and Lindberg were kidnapped from their home. Was kidnapped from their home in east Amwell, New Jersey, March fourteenth. George Eastman, the inventor of the Kodak camera, shot himself in the heart, aged seventy seven. March seventeenth,
the German police rated Hitler's Nazi headquarters April's second. Famous aviator, Charles Lindberg, paid fifty thousand dollars as a ransom for his kidnapped son. In April nineteenth, President of the United States Herbert Hoover first suggested the five day work week.
So you can.
Blame him for that. What was it before seven days?
Seventh? It was huh, the were six actually because he was supposed to rest on Sundays, right, okay.
May fourth American gangster Al Capone entered the Atlanta Prison convicted of income tax evasion. May fifth, Japan and China signed a piece treaty. May twelfth, the body of famous aviator Charles Lindberg's kidnap son is found in New Jersey. Ransom didn't do any good, apparently. May twenty first, Amelia Earhart became the first woman to complete the transatlantic solo flight, having flown for seventeen hours from Newfoundland, Canada, to Londonderry,
Northern Ireland. June thirteenth, Great Britain and France signed a peace treaty. July twenty eighth, Douglas MacArthur, acting against US President Hoover's orders, commanded several attacks on the Bonus Army, which was World War One veterans in their families, attempting to a victim from their encampment. At least two veterans
died in the attacks, with fifty five injured. July thirty first, Nazis gained thirty seven percent in the Reichstag elections in Germany, becoming the largest party in parliament by a large margin. August second, American physicist Carl David Anderson discovered and photographed a positron the first known anti particle, and Thomas, do you have some other news about physics, don't you?
Well? I saw that the nineteen thirty two Noble in Physics went to Werner Heisenberg. Ah, he got it a year before Schroedinger. So all your quantum mechanics and atonic theory.
A lot of stuff happened in thirty two, all right. So August sixth the Venice Film Festival, the world's oldest film festival, opened for the first time. August thirteenth, President von Hindenburg refused Adolf Hitler when he asked to be appointed as chancellor, and instead offered Hitler the position of vice Chancellor of Germany. Hitler refused a position and announced he would oppose every government not headed by himself until
he was chancellor. August sixteenth. John Lindberg, Charles Lindberg's second son, was born just five months after the kidnapping and death of his older brother, Charles Lindberg Junior. Almost done. Here is Stember twenty in his cell at your word. At jail in Pune, India, Gandhi began a hunger striking against the treatment of India's lowest classes, known as Untouchables. October two, the New York Yankees won their twelfth consecutive World Series game.
Yay.
As a Red Sox fan, I can still appreciate that. October third, Iraq gained full independence from Britain and joined the League of Nations. October ten, the largest hydroelectric power station, the niper Dam dni nieper nieper Dam, was first put into operation in the USSR. It's actually Ukraine, but well it is now, right, always was was it USSR?
Then? Well, okay, he was always in Ukraine. Okay.
November sixth, the Prime Minister of Italy, Benito Mussolini, introduced an amnesty decree freeing thousands of convicts. December fifth, Albert Einstein was granted a visa to enter the United States, and December twenty seventh, Radio City Music Hall first opened in New York.
Sit.
And that's just a little bit of what happened in nineteen thirty two. A lot of very famous people that you know were born in nineteen thirty two. But I'm not going to go down there. It's sort of the threshold, right, like that's a long time ago now.
Yeah. Yeah, I'm working on the space geek out, which will have already been published by the time you heard this,
and I'm still writing the script. And nineteen thirty two is also when Karl Janski created the first radio telescope, not because he was an astronomer, but because he was trying to build a directional antenna and he kept having this hiss no matter where he pointed the antenna, so the directional antennae could tell which way like thunderstorms and stuff were because lightning strikes have created a radio wave that he could say, Okay, well the lightning coming from there,
but there was this hiss that was coming from everywhere. Wow. And it took a while to finally figure out that it was the hiss of the universe. Yeah, and it's actually the cosmic background radiation from the Big Bang. Wow. And that he ended up building the first radio telescope just trying to figure out what was going on, an entirely new class of astronomy. So cool.
Yeah, you got to have an imagination to do that kind of stuff, can't.
Just like he had a problem. Yeah, the problem is what's that noise? What is that? Okay, Richard, who's talking to us? He got a comment, Yeah, grabbed a comment on off the show eighteen fifty eight of Moth we do with Thomas back in August of twenty three, so a little over a year ago, and that was the leveling up your architecture game conversation. Seem to have a theme with your shows, Thomas. And there was a bunch of good comments on the show, and this one comes
from Laslow. He says, the more shows I listen to about software architecture, the less I'm sure about what it's about. I've seen great examples of software architecture and designs, but I've never seen them evolving from an architect's point of view.
A big bowl of mud never starts at the beginning of a software's lifetime, but a couple of years later, when the architect has left the project, new developers are not fully onboarded, and the clients realize they can push your requests that were previously quote too hard or too expensive to implement you and sometimes junior developers cross the line in previous design reels to be clients happy, making the application architecture rot. I'd like to see an architecture
that stands the test of time. Hm hmm. You know, architecture isn't a thing right. It's a set of ideas that have to be implemented by people, and as long as people don't implement them, you can take any building and stick something horrible onto it and damage its architecture. Same with any piece of software. The only way architecture stands the test of time is that people choose to care for and pretty much leave it alone. Well, no,
I mean challenge it, improve it. I could. It makes sense when you have a feature that's a problem in the current architecture, say, is the architecture wrong, Like, is there a way to press against the architecture to still get some of the benefits of it, of the extreme design? Well, we do that all the time anyway, and some of us just admit to it is compromise, right. No, you know, all ideal architectures can't be built. It's always a depression of less than I do.
You always end up with an architecture. It's whether you made it explicit decisions or implicitly accepted the decisions along the way. And if you just keep letting it rot, as they said on the comment, then yeah, you'll end up with the big ball of mud and you won't have a nice, clean architecture, but you'll still have an architecture.
Unfortunately. Yeah, nobody makes a ball of mud, it just emerges. So Las Well, thank you so much for your comment. And a copy of music co Buy is on its way to you. And if you'd like a copy of music code By, I write a comment on the website at dot net rocks dot com or on the facebooks we publish every show. There to be comment there and I read it on the show. We'll send you a copy of music code By.
And we're also on all the social media's we're on. We've been on x Twitter for a long time, at Carl Franklin at Rich Campbell, and we're also on blue Sky. I'm at Carl Franklin bsky dot app, and I'm Richcampbell at besky dot app or and is it app? Is it app or is it social I don't even know whatever blue sky.
Dot, bsky dot net or dot whatever.
Yeah, bsky dot app. It is bsky dot.
Okay, good because you know, I just pull it up in the browser and it's there.
I don't really pay attention much anymore of the URL. But anyway, doing all that stuff through buffer these days, I you known't actually see the regular sites much. What's buffer pray tell She's a tool for looking across all the different social media's that you're dealing with, and that's how we get all the posts out for all the shows I need that. It's a good little product. Yeah, nothing bad to say about it. I switched to it attle while ago, and I've been very happy and it works with them all.
And we're also on macedon. I'm Carl Franklin at tech Hub dot Social.
And I'm rich Gamble at mass it on social, So.
You know, get in touch with us, ask us questions and you may get a free copy of music to Code by as well. All right, Well, that last voice you heard there before Richard was Thomas Betts and he is a laureate software architect at Blackbod baud Blackbod, the leading software provider for social impact. In his spare time, he contributes to InfoQ dot com and helps organize Q on software development conferences.
This is interesting.
Credits dot NetRocks for inspiring him to give back to the software community as a writer, podcast host, and international speaker. Well that's nice. How you doing doing well?
Yeah? I realized that we don't talk about the stuff I do with InfoQ, but it's kind of my side project. That's the job I've had the longest now for like eight or almost nine years.
Wow.
So yeah, being able to speak locally at a developer conference and now I spoken internationally and organize a conference. So kind of following you guys the example. You know, it started with writing into comments I think, like I don't know, two thousand and seven. Yeah, just to join the conversation and give back to the community.
Yeah.
Yeah, we've been friends for a long time.
Yeah.
And so your your topic these days is architectural intelligence, which you say is the next AI, meaning the next AI acronym, right, exactly.
So let's define what that is.
Well, I want to you know, right now, we've got everyone's wanting to put AI on everything. But I think we're in that Arthur C. Clark moment where any sufficiently advanced technology is indistinguishable from magic. Oh yeah, and that right now, any software we don't understand, we just call AI.
Right.
We don't know what it is, but everyone's asking for it. The CEOs are asking for it. The product owners are saying, I need a AI in my products. We look innovative customers aren't asking for it, but give it a couple of years and the expectation will be there. Like, why it doesn't this have AI? It must not be modern?
So Thomas real quick. I play in a band and it's a ten piece band and it's awesome, and you know, after we do a particularly great tune, I will say, this band does not use AI.
Yeah.
It's kind of like when you get rice cakes and they say fat free. You know, it's like, well, duh. But everybody thinks that anything like any kind of pedal that we're using to change our voice or any kind of sound. You know, software is AI just because it's good and it might use digital signal processing or whatever, but it's not AI.
Right.
I think we call stuff AI until we have something better, and then we call it computer science.
Ye.
Right, we have a product and a tool and a name, and if we go back to you know, like AI. When we talk about AI, mostly what we mean is generative AI, not general AI, not artificial general intelligence. That's data and the terminator and other characters from science fiction.
I don't think the average person even thinks that far. They're just looking at large language models right right.
Right right, and genhi is large language.
Model is one of them. But you're doing gen ai disservice if you can, you know, keep it scoped into LMS. It's a bunch of other things. It's been you know, that's been going on for a decade. LLLM is only kind of detonated with chat GPT in twenty two. Like this is all. This current storm is pretty recent. And anything I've learned from the shows we've done recently is that the actual smart machine learning people in this space
are pretty offended. What do you do it because a lot of this stuff is sloppy machine learning.
Yeah, and I'll I'll just talk about jen ai. In terms of images, I can spot a chat GPT generated image a mile away now.
It just has a look to it.
And I'm I'm offended when I'm scrolling through Facebook and I see a picture of this idyllic house, you know, with perfect lighting with waterfalls going through it, and you know, there's no comment. It just says ah or something like that, you know, and and there's a million views and a million likes, and it's clearly generated. It doesn't exist, and there's no there's no place, there's no date. It's just like a bucolic setting, right, and.
Half of those million likes are the that are liking the thing that the other bot created, so.
And people that use them to take pictures of themselves and turn them into AI pictures.
No, stop that.
I like that you brought up like traditional machine learning, like we used to call what's now just established machine learning and an mL model. Like for a while that sounded like AI, and then we move the AI a little further. It's that marketing term that just kind of is the umbrella.
Yeah, I've always said it coined it as artificial intelligence is what you call it when it doesn't work. Yeah, so as it does work, it'll get a new name.
But I think there is that correlation between a large language model and other machine learning models, Like the difference is in the algorithm inside. And if you're not a data scientist, you probably don't understand. But I still, because I don't understand the inside, I treat it as you know, a function box. I put it in an input, I
get an output. So if I'm doing image recognition, I send it to my image recognition machine learning model, and it says this is a cat, this is a dog, or comments on the on a block well.
And it's the whole point. I have an API. I don't want to know. I don't know how to create a cryptographic key, but I do know how to call an API that gives it to me exactly.
And so these are the things that software engineers know how to use, Like this is some little function box and I can just call it, and here's my input and what's my expected output. I think what you need to understand with large language models is you give it a series of tokens, a bunch of words. We call them tokens, but token might be part of a word. That's some of the semantics need to learn.
But I think it's a you know, the joke is a tokenization. I think it's the cleverer attack here, right, Like the stochastic parative spitting language back is not that impressive, But the fact that we've come up with a strategy for converting language, virtually any language, into a set of numeric symbols that, by the way, cross between each other, like it's the Babelfish man, Like you've almost cracked universal translation.
Right, because that's that's what tokenization is. Like, here are all I want to take this string of characters and turn it into a bunch of floats, and it's not the asking number of like this is an A and this is a B. Like this word or part of a word has this representation in a multi dimensional array, and that's all it is. It's a lot of math, right,
it's all statistics. And when you feedsie into a large language model, here is my series of tokens, here's my words and my sentence, and all it gives back is one. It just predicts the single next word. And I think that's the mystery people don't recognize, is it takes that one like, well, that's not useful. And you watch chat GPT and especially the early versions, you watched it type
out very slowly. What it's doing is it's feeding that one token back in adding to you, and that keeps building up the context that auto regression eventually produces a series of words that say, oh, the next part of the sentence is likely to be this, yeah, and it looks like magic yeah.
Well and them Importantly, it can be interpreted as intelligence where none exists. Yeah.
And this goes to if we look at the you know, the learning aspect of it, like we're fine calling it machine learning because we start with training data. For any machine learning model, you give it a set of training data. Maybe that's your sales data for the last you know, quarter, or it's pictures of dogs and cats. In our case, it's just words. Have this thing read everything you can
find and train it on that. So you give a lot of training data that's words, and then it's really good at understanding words, but it's still just predicting the next word. And when we see something that looks like a human probably did that, and we don't understand how a computer could do that, we think, I don't understand it, it must be intelligent, and we're applying that where it just doesn't exist.
Well. Plus, and humans are prone to that sort of thing anyway, right, Yeah, heck, we think our dogs understand Yeah, and we talk to our cars, which especially weird this tendency to ANSWERPROMORPHI. It's like, it's oh, it's necessary. I mean, is it necessary or is it a weakness?
Oh?
I think so.
I think it's necessary for us to be able to have some sort of relationship with the thing that we're anthropomorphizing. It's easier for us if we give it human at attributes. I think it makes it easier.
Architects are good at coming up with metaphors to describe stuff and people. How is someone going to relate to the software? How do I translate this complex idea into a design that my engineers are going to be able to implement or the users are going to understand? And things like the desktop on my computer, well that used to be analogous to the desktop where I'm setting the computer down on and I can have a pile of papers here.
Right.
We have these ideas and then those just become things, and then a floppy disk icon sticks around for twenty years after we've gotten rid of floppy disks yep.
Yeah, and now it is more better known as a save icon than it is as a physical thing, right. Yeah. People think it's weird that I three pre printed as save icon, like what's wrong with you? Why would you do that? But everybody recognizes it and that's why we still use it. Yeah.
So let's get to the architectural intelligence part of this. I think it comes down to two questions. If we have jen AI or really just LLMS, the two questions are is this appropriate for my software like the scenario I have? And then if I decide it is how do I optimize it? And I think you can look at some of the examples of when it makes sense to using your software because it is a language model. It's good at doing things that are language based, right, right,
like you want to have a natural language interface. These used to be things that people worked really hard on and turns out you can just throw stuff at it and it can translate it into something else. Like you said that universal translator. I can't write a good search query, but now I can just talk to it and transcribe and it just works.
Yeah, and you might think of replacing certain complex patterns whereas you know, picking something from a series of drop downs and then maybe you know, in a grid in setting some things where you could just pop up a box and ask the user what they want, you know, what they want to see or what they want to do.
Yeah.
My company has a hackathon once a year and one of the teams that won actually people I worked with. They were looking at our custom ad hoc report builder, right, very customizable. You can choose all these things, and they realized that all you're doing in the UI is sending requests to the API to say, create a report with
these characteristics. Yeah, so they train you know, basically wrote a prompt that says, here's the API, and you could ask it please create a report that runs every Monday morning and the week over week totals and gives these values, and it knew what to do and you simplified the complex part of the system, but you didn't replace the actual report generation, right right, just that interview.
It's literally a ux change to UX change. Yeah.
Yeah, And again it's that language level of the UX change. The language model is good at understand language where it starts to slide into the maybe we should or maybe we shouldn't. Like if you're asking for in product help and you want to find stuff like maybe that's a little better than searching your help and maybe and consummarize the results. We've seen stuff like that. This is where
RAG comes in and does different things. But you know, some of the examples I've seen like, oh, we have a rules engine because we have all this complex logics, we added a rules engine, and now we have a rules engine to manage, and now we have difficulty figuring out like what are the rules that are applied? What if we just replaced the whole thing with an AI and it can just do the logic the rules engine.
This I think goes to the example you give for it Air Canada that basically said someone's eligible for a discount, Yeah, because the AI said so, But there's no actual rule there. It just said the most likely answer, the most likely next word when you ask, am I eligible for a discount? Yes, it's just predicting a word, but it's not actually based on logic.
Yeah. And more importantly, when then that went all the way to court because it's like, oh, our software failed, we're not liable, It's like, no, you presented that as a replacement for a human agent, and if human agent had said the wrong thing, you would be liable for it. And so you're liable for it, right, which is good. You know, let's get some case law in place there, and also pressing against employers to say, if you're doing this kind of utilization of this software, it comes with
a price, so you know, test carefully, right. I think that's the bigger issue I have is a you know, I mentioned that the API for calling cryptographicky, but what if one in one hundred cryptograph keys were just invalid but you had no way of knowing like this is the problem I had with lllms is you're putting them in a critical workflow and you haven't really tested them. You don't know what the failure modes look like.
Right, And I think anywhere you're going to use an LM, take the LLM out and put a person in that place.
Yeah. Right.
And if you asked a person to do something, would you trust them or do you have other checks and balances? Is something has to be reviewed by a supervisor or do you just let that person have full autonomy and they can do things or you have later auditing to say, hey, they messed up and we can fix it. But you can't just magically think that this LLM can do everything because it can write poetry.
So, guys, as if United Healthcare wasn't already in the news enough, they are facing a class action lawsuit alleging that the company misused AI to deny specific insurance claims, and especially on elderly people. So there's you know, there's a I'm going to link to a news story about this, and there's an interview with people who were denied and clearly weren't shouldn't have been denied. An insurance claim and they basically suit and they said, yeah, your AI basically made this determination.
Well I think that so, yeah, but they all so configured it to do that, right, It's no different than having a person say deny all acclaim.
But I think that's also where we're seeing AI is thrown on as a label what was probably just machine learning. Right, they have all this could be historic claims data, and there are plenty of examples of it is.
Actually it was predictive analytics that they used because the ices are pretting that if we allow this, you know, the chances are that they're going to I don't know, abuse it or whatever.
Yeah, those those models are trained with biases because it's the day you gave them, and if the only data you gave them. I think the crime ones are horrible, Right, It's like, oh, there's all this crime because there were these arrests in these places. Therefore we're going to send more people, which then leads to more arrests because that's where they sent the cops. It didn't actually solve the problem of why is high why is crime high in these areas?
Right?
Or we found that where there are house fires, there tend to be a lot of firemen, So let's get rid of the firemen.
So I think we're getting into my third category. We went from good to maybe to the really questionable uses of AI. This is where people think I should just you know, replace that report generation. If you need specific mathematics like added up your regulatory and compliance scenarios your quarterly reports, you really really really shouldn't do that because it will come up with numbers. They're not going to be the right numbers. It's based on statistics, based on
words and language, not this is my accounting model. We've had accounting standards for like four or five hundred years for good reason, Like we know how to do math. Don't put it in those places.
What if it can help you generate the query.
Again, that goes back to the where does it fit? Put it in some place. That's the language aspect. That's what it's good for language aspect. Yeah, But I think what people need to understand is this is non deterministic software. Right, we are used to software is a series of if then else statements.
Right, very deterministic.
Yeah, AI is very non deterministic. It might give the same thing, but it might give something different. And this is where you can set temperatures and all the different things and move the sliders around. You can get more creative answers or less creative answers, or more specific, and you can get it to repeat the same answer. But it's always still based on a prediction model.
The fact that you can get different answers to the same question, it's just just clear indication. It's like, it's not that determined. Yes, the determined model will give you the same answer for the same question.
Yeah, I got to admit that I have used at GPT to generate store procedures or SQL queries where you know, I gave it the the data that I needed, and you know, I'm just I'm not the sequel guru that Richard is, and I don't know, you know, my group buys completely befuddle me. And it worked, you know, it turned out to return the right stuff.
So you just said, you know, you gave it a sample of your data, like here's what it looks like. The more specific you can get, then the better the answer is. It will always give an answer. It will never come back and say I have a question, can you provide me more information? If you said, please generate a store procedure it would write one, it'd be completely useless for you. But if you said I need a store procedure that gets this data, that prompt is giving
it more information and that narrows the context. Like AI is really good at broad general statements. It knows all this generic knowledge because it read the Internet. It doesn't know your specific scenario. And that's where architects come in. Is like architects take these design patterns we have and we figure out what is useful in this scenario and is AI one of those things that might be useful in my design at this time or does it not apply?
But in this case, it knows the rules of tseql and you gave it everything that it needed in order to create the right select statement and it worked. Because of that narrow scope. You also have a testability aspect there too. Were able to try it and evaluate the results and decide if that you know was correct.
The compiler gets to say yeah, as they say too right, like if the if the database didn't like it, it would have spat it back.
I also like the ability to if I'm asking it to generate a method in c sharp that does X, and it smells a little funky, and I would say to it, can you try this again, but with less verbosity, can you maybe use link or do something like that to you know? And it'll say sure, and it'll try
it and it'll work. On the other hand, I've had methods that use link and a lot of complex link and then I will say, hey, can you expand this to use loops and if then statements, And then I will take that and I will I'll test it, make sure it works, and I'll comment that for somebody who's reading it who really doesn't understand link, say this is what this link statement does.
Yeah.
I just had to give a demo of get ub copilot within my company and it was a five minute lightning talk. I said, we had this new feature we're developing. It was pretty complex, so we spent a little bit more time than usual doing a upfront design. Just wrote out some markdown, a few mermaid diagrams, here's some classes, and here's the API end points. And I'm like, what would happen if I just gave this to get up copilot And the first time I did it it wasn't great.
It worked, but it wasn't it didn't It didn't fit the style of the coding that we had in our project, things like we used filescope name spaces because who needs extra curly braces. And then I changed my prompt and I said things like, please use filescope name spaces, and please create the interface for each of the classes and injected into the controllers like things that a developer would have known to do based on the design document, because they said, Okay, you only need to provide this level
of detail. I can figure out the rest with Copilot. It could do better once I told it, please follow our guidelines, but I had to tell it what our guidelines were.
Yep, just like a developer, same thing. Yeah, give better and better instructions, get more precise results.
Yeah.
Hey, I think it's time for break. So we'll be right back after these very important messages. And if you buy chance do not want to hear these messages in the future. You can get an ad free feed by becoming a five dollars a month patroon at Patreon dot dot NetRocks dot com. We'll be right back. Do you have a complex dot net monolith you'd like to refactor to a microservices architecture? The microservice extractor for dot Net tool visualizes your app and helps progressively extract code into
micro services. Learn more at aws dot Amazon dot com, slash modernize, and we're back. It's dot NetRocks. I'm Carl Franklin, that's Richard Campbell, hey, and that's our friend Thomas Bets and we're talking about architectural intelligence. Should I shouldn't I? And if I should, where and how much? And why do we even need to do this?
Yeah, I think we were leaving it off with AI being nondeterministic software, and I want to get the idea that that's a feature, not a bug.
Right.
It gives these good enough answers, like that's why it seems intelligent, Like it did a really good job. And there are times when the really good job is flat out wrong, but there are times when it's going to be okay. And find those places in your applications where it's like I can tolerate the good enough answers, Like if someone doesn't find all the help references, but I found enough and they got their job done, that's okay.
Yeah. I like your reporting API wrapper scenario because you're experimenting, You're trying to come up with a way to look at the company's data in a way that'll presumably allow you to take an action, so you don't know exactly what you're asking for. The most frustrating thing I see with most people play with some kind of report builders. They gather get everything or nothing right. They always have
scoping problems and so forth. So a tool that allows them to get to maybe do a little more iterative and improve an expression on that that might be an easier way to go.
The anthropomorphizing problem is a big one, and Richard's been banging this drum for a long long time. Don't fall into that trap, or try not to. But it's kind of like having an educated uncle who sounds very smart and uses big words and never says um or like or you know uh, and you know you ask them the question, they give you a very intellectual sounding answer, and it may be completely wrong. They may have brain damage.
You know, your uncle may be educated at Havad. However he got in a car accident a couple of years ago and hasn't been the same since, but he's still sounds very smart.
I like, I've taken issue with some of the terminology like get hub copilot, great product name, it's not a copilot. Like we talked about this as being your AI assistant. It's going to help me do my job. I've flown on enough commercial flights they all have co pilots, and I'm pretty sure those people are fully qualified to fly the plane and are probably doing it. Also, on a
long flight, they aren't flying the plane. They flip on the autopilot, So at some point you are trusting the computer to do the thing with monitoring.
I think Getthub Copilot did a disservice to real copilots. Yes, although I did appreciate the name at least implied to you, Hey, you're still the pilot. It's still your fault.
I think AI agents has been like changed to a gentic AI, which is better but harder to say. So no one says it.
But that's the idea of the disease.
The AI gets to make these decisions and we're not there yet. No, Like we shouldn't just let them run a muck.
I don't want to. I don't want to get there. We're going to get there. Yeah, it's going to have This is the thing they're pitching now, and there's certain workflow. I mean, it's really not that different from any sort of stream based bit of software that has the ability to to act in some way, right, Like, I've played with plenty of prescriptive analytic models for email prompting. You know, you don't necessarily you don't want to be in the workflow of this person's been to the site, they put
in some stuff in the car. If they didn't buy, we send them a tiicle email about the things they put in their cart, right, Like, all of that is automated now, and the fact that it's using a machine model to determine when to send that email, like, don't send it right away, that's creepy, right, it's a few hours later, and they're actually using all of the relative response data to feedback into the model to adjust the time.
Ye.
Now that these agentic models are going to play on that presumably take it further and maybe make it easier to build, because good prescriptive modeling is hard, Like, that's a tough thing to build, So maybe we're lowering the bar for how to make this stuff.
You also have to beware of AIS or AI agents rewriting things that you wrote and making sure that if you do choose an augmented version that it's accurate, so that means you have to you right. Let's say, let's say you're on Facebook and you want to do an AD or something, so you write the ad copy and then it gives you three other options. Hey how about this, which sounds one of them sounds more exciting and stuff,
but it left out details. It left out links, for example, it left out references to other Facebook pages, so makes me skeptical.
Yeah, one of the other. There's actually a product that's out there now for my company because we do software for nonprofits, a lot of fundraising, a lot of donation, and the people who are good at running a nonprofit and doing the behind the scenes work might not be the best people at writing the message that little, you know, short text it's going to go out and say, hey, please donate for this cause or the we have just giving, which is kind of like go fundme. You have a
small little thing. How do you put that little blurb out there that's not too long that gets people's attention. And so we have tools that allow you to, you know, click these buttons and change the tone and it'll help you write the message. Because again, it's just language. It's great at writing language that convinces people, and we've seen people donate more to causes that are using that product.
So you know, there's a very clear correlation to well, it's good for our products, good for the nonprofits that are using it. So again, find the right places for it. I think the agents they're going to get better as we get them to be more specialized. And this is one of those weird things that the bigger is not always better. We've talked about these lms, right that have
grown to billions and billions of parameters. Right, I can't remember what GPT three to GPT four like double the number of parameters, basically how big the model is and how many things it knows.
It was more it was like one hundred and seventy five billion to a trillion. Yeah, yeah, you know, five times and the context size growth.
Right.
Yeah, Well, there's an argument there's no path forward because there's not more data, Like there's not four trillion parameters to be had.
Yeah. Yeah, they can't train it on anything else because it'll start training itself on what it knows and then it'll be the cyclical. It'll just like go down the biospiral.
You know, in science fiction when we talked about a superintelligence is that it would start self learning and getting better. So far, this software when self learning gets worse, like if you if you put it back out to train against its own data, it gets less effective, like it's a photocopy of a photocopy.
But for the average user, they can build up a context over time so that it quote unquote no, or remembers things that you've talked about in the past. So you can just say, hey, remember that application I was telling you called blah blah blah. Yeah, I have a question about that, and it'll know well quote unquote no, yeah, it'll pull up that context.
And that's the most effective way to use any LLM is good prompt engineering, And that's like we talked about rag go and find this data. But you've got to have someone who knows how to do that effectively, because you know it poorly, you get worse results. Here's how to index my data. But if you can stick everything that you could possibly want to know about this question
into the prompt, you're gonna get great, great results. The problem is the corpus of knowledge that I need you to know about for my product won't fit into forty thousand characters or thousand.
Now if either of you run into Windows Recall yet not yet.
We have a new laptop that has it, but I haven't played with it yet.
Yeah, I mean there's been a lot of This was an announced back in I think Microsoft did a terrible job of announcing it was only for the new laptops, the Copilot plus Com based coplout plus PC laptops. But it's basically taking a snapshot of everything you're doing the whole time. Like you talk about a way to generate knowledge about you for a system I could see, you know, at the moment, it looks like it's largely just a search engine. Hey what was that pair of pants I
was looking at in the past? Right? And the fact that it can sort through all of that because it has a copy of everything you've done stored on your machine is interesting. But there is that larger idea that over years, you'd gradually be building a remarkable augmented retrieval set about yourself just because it knows everything you've done,
given that you were only working from one machine. Because it doesn't and it's only stored in that machine, so you're within the constraints of what that computer can do in the name of security, but if you did anything on your phone, it's not going to know about it. If you have more than one can who would have more than one computer? I was drained? Well, Gmail is that experience for me now?
Yeah, I mean, and it has been for a long time. It's your store of intelligence, it's my store of intelligence. But the problem with it, of course, is when you want to find something, you're going to hit on all of the spam that you've gotten first, and because there's more of it now, you have to really get creative with your search, your advanced search, and you know, that's kind of the problem. Whereas this recall thing sounds like it's a little bit more focused, and.
There's lots of people freaking out about the security around it, like, oh, of course, and they were as soon as it was introduced.
Yeah, I think any one who's taking this seriously understands there's a security aspect. I think you're going to see small language models become more popular because there isn't it's not going over the wire. I can the model is small enough. I can host it on my laptop, or I can host it on my servers. I don't need to be calling out to chatchy here. Open ai APIs right, and so I don't have the concern of sending the data over the wire if I can self host it.
Now I'm paying the hosting costs of hosting a model, but maybe I get better results because I don't have to sanitize my data. So I'm able to ask better questions because I know that the data is not leaving my domain.
I don't think the customer cares at all, right, like they're happily using chat GPT inside of companies even though they're specifically forbidden. I'm doing so, you know, on the administrative cybork are battling. Give them a path forward to use actually secure approaches. Right, people cared about security, we'd have a lot fewer problems. There's nothing you can do to me. People care about security. Cost there is something, right.
Yeah, if you can if you can equate security to dollars, like if we don't take these secure initiatives, we're going to lose x amount of dollars, then you'll get some of these attention.
Right. If you run this in the cloud, it costs you, you know, and dollars a month. And if you run it locally, it doesn't, right, you might actually win some folks over to Yeah.
I think that's why Microsoft copilot. Again, the copilot name is now like any different products. So it's a biggest I've heard internally. It's over two hundred sure, wow, But it's the I want to search our SharePoint in one drive and teams and everything else. But it's like, here's the thing that we provide, like well, all the other stuff that we trust Microsoft to secure, that's going to take care of it. I'm sure Gemini is going to
do the same thing. If you're a Google based company, right, you're.
Doing Google workspaces, you're doing the same sort of thing more or less. The uh yeah, and I think you're talking about M three sixty five copilot in that case. But it looks like there's at least several flavors of M three sixty five copilot. The thing that'll give you a hints to make better PowerPoint slides is different from the thing that'll find stopping SharePoint from you, but it's all under the M three sixty five copilot matter.
So I got bills from Microsoft co Pilot three sixty five and I never signed up for it. Did Dick automatically convert your office three sixty five subscriptions to copilot and tack on a couple hundred dollars for some reason.
I don't know the answer to that.
Yeah, anyway, you mean an AI to understand Microsoft billing.
Yeah, I guess. One thing I want for a lawyer.
One thing I want to mention to you, Richard is we should get test Ferrondez on the show talk about RAG. She has some different ideas than mainstream.
Yeah, there's good tech in that space, no, no doubt. And it is the sort of balance between consumer level tools for this and this line that we play in developers where we're building code for our companies and they have other requirements, but it's also utilizing the data of the company.
I mean, I would be highly resistant to wanting to build apps in this space with things like M three C five copilot around, Like when it comes to navigating through the data of the company, that seems to be the tool given you're in your company is in M three sixty five. Yeah, right, But the I mean, I wonder if it's just a UX feature when it comes to l MS, it's really just a UX feature. It's a different way to communicate with a p software to get results you want.
Where we're at right now, that's the best use case. Everything below that starts getting into the questionable should we do it?
Or is it good enough? And I think we did a show with vishuwaz On that long ago where he's getting involved with a startup where it's trying to write. It's trying to use the tool to write proposals more quickly than humans can, except the company then has to submit that proposal and comply with that proposal. So making sure that proposal is accurate is not a trivial problem. In the end, writing the software to gender proposal easy part. Testing to make sure the proposal is correct and is
capable of doing what capable of executing on it. That's much harder and what takes longer for somebody smart to write the proposal himself or for somebody not quite as smart to generate the proposal and then have the smart people read it and make sure it's right. And the
consequences of being wrong are is money right? You're going to submit a proposal for a you're not going to be capable of doing, or if it's grossly underpriced, or you know you missed a requirement, Like there's validating these things is not a trivial problem. I look at it almost like writing contracts like I've had. I've dealt with companies where we had service level agreements with other companies but had no way to measure where that we were compliant.
And the problem is the level of effort to write a good contract and a bad contract is exactly the same, more or less.
Yeah, if you're.
Pressing the button for putting this in and the AI generates it, you don't know if it's a good or a bad one until you have an expert review that. And so you get the perception that, oh, we've saved this effort because look at how easy it was to press the button. But if you still have to have someone do a thorough analysis, and maybe it made it easier for them to do the review than to be writing it the whole time, and maybe they make a mistake, and so you still need a second editor to review it.
Maybe you only need one qualified expert instead of too. But there's that perception that, oh, it's it's good, and you don't realize that it could also be just as bad. Yeah, And if you don't know how to tell that this was good and this is bad, then that's not a good implementation.
You can get back to this. You really can't use this tool unless you're qualified to have done this work without the tool, right, because you need to evaluate its output.
Yeah, Like I said, if you take the LLM out and put a human in that, what would you do to make sure they did it right? And we tend to do that because we don't trust people for really important stuff like I'm going to write this contract, Richard's going to review it to make sure that I dotted all my eyes and cross my t's.
I would argue the LM's better in the checking role that you get the contract and then you run it through the LM to say what's been missed.
Yeah, and that's like having copilot and GitHub copilot like write the tests.
Yeah for my code?
Should you have it write the code and write the tests?
Well?
Maybe maybe not. I have the developers do that as well, so we kind of accept it.
So a little that's unnerving because it's like garbaging garbage out and it's like I couldn't write good code, but I can't write good tests either, So it passed. Well.
There are people out there who can read and understand code, but aren't so good at writing it, especially when it comes to architectural decisions, right, so you know that may be a good role for them.
And that's where the context matters, Like the architect is always going to say, it depends. So how do I know that this code is good code in this application? In this instance? So I can have it write the code, but is it the right code? Does it follow our patterns? And that got back to the idea that I learned. If I give it a better prompt and say here
is how we write our code. Follow these standards, but that's still just basic stuff like how to write async and where to put your curly braces, it's not follow these design patterns.
This also reminds me of how outsourcing actually was the mistake, because if you can't describe the problem well outsourced, it's not going to work any better if you did it locally. But as we got better at describing problems for U source work, we actually got code results. You get better at describing the in the prompt what I need you to do here, you know through that tool you're going
to get usable results. I do like the idea of these tools are good at making going down the checklist, like taking an architectural design and feeding it to an LLM with a good prompt about what our expectations are around an architectural design and saying what have we missed? Like what would you correct?
There's one thing I really hate about being the recipient of LLLM generated content, and that is being lied to. And I get an email that says, you know, hey, we looked at your podcast. We think it's amazing. We want to talk to you about blah blah blah, and I really love this episode with YadA YadA.
And you know it's just all generated.
And or here's another one I got recently. We'd really like to buy your company. We'd really like to buy your company. We think it's interesting and it's whatever. So I ignore it because it's obviously a bot. Yeah, and I send another email another week, Hey are you are you really? Are you interested in selling your company? Ignore that another one ham circling back. You know, blah blah blah, you kind of And so I wrote back and I said, why don't you tell me what you know about my company?
Cricket of course not. They just the whole goal was to get you to respond it anyway, and then you just go on a list. Now that person will respond.
They ignored it, you said saying about having it asking it to analyze your design documents. I think that's something that the architects can really benefit from, Like we can use these tools to our benefits. Another pair of eyes. Yeah, you know, metaphorically speaking, use it as your rubber duck.
I don't always have someone around. I've got a whiteboard here, but that's you know, for me to like draw stuff out, and I want to know, is this good If I write up in an ad R an architecture decision record what did I miss? And I can ask it to you know, analyze that and find things messed.
It's another thing the tool is good for is documenting what we did, right, Yeah, recording the meeting where you generated that ADR in the end and then having the tool summarize that as part of this is why this ADR. There was a meeting on this date. These are the people are there, These were the key talking points. This was the decision ADR fire the secretary. You're supposed to write that up. We just don't.
Yes, yeah, we had the meeting, we made a decision. Yeah, but the why behind the decision gets in the discussion. Yeah, and so if they can capture that. And then there's also the I need to communicate with different people. This is the architect elevator idea. I need to talk to the CEOs and the CTOs all the way down to the engineers and the basement doing the work. Those have different audiences. I might have that same design document that
I need to convey in different ways. I need to get more detail to the engineers, but I need to summarize really quickly.
I could see as an architect, you'd build up a body of prompts. It's like, I'm allowed to take this adr to the CFO. Here's the CFO prompt. Yeah, they care about return on investment, they care about initial capital costs, like include these numbers, like that kind of thing, so that the tool would spit out a fairly well shaped thing, and you really use that over and over again. Yep.
Yeah, I think we're going to see the I have my toolkit of here's the things I have, and I used to write macros am I going to write those those prompts that I reuse that stuff that I had to do that one time and said please use filescope namespaces. I shouldn't have to tell it every time, but it doesn't remember, so I had to tell it every time. But if I can have a macro that starts that, like start our code with this and it's pre injected, that's useful.
Yep. Yeah, Lank, you're language scoping now. You know language of the Executive Committee, language of the security group, you know language of the vendor. Yeah, you know. I I have gone back with and re read previous interactions with a given vendor, like an ISP that I was working with on a project. They say, you know what, what were the ones that worked? Essentially when we talked this way, we got better results and you were almost cut and pasting from previous ones, like in a way building up.
That prompt would be that same example, maybe a little quicker and a little clearer. I would also say, as someone who organizes conferences, I can tell when you use cht GPD to write write your abstract. Yep. So that's got seven hundred submissions in three hundred of them, the same opening sentence.
G I think this is one of those things that QCon stands apart, we don't have a call for papers right as one hundred percent human curated content. We get together like six months in advance. I was on the program committee for QUCAN, San Francisco, and we say, what are the topics right now that I want to learn about that I think other engineers want to learn about that.
Gave us our twelve or fifteen tracks. We find track hosts for each of those, and then they find they reach out to their network and find five people to talk about this topic.
And that's great.
That pole model means you're using the intelligence of the people who know what's relevant right now. It's not a push of police accept my talk.
Yeah, yeah, doesn't mean they didn't also write the abstract with the machine.
Well, I mean yes. I expect everyone to be using whatever tools are at the disposal. And if if an AI and an LM allows you to make a better presentation, that's great. I like the presentations that say I used Claude to help write this presentation. Here's where it sucked, because that's useful information to me, Like it just can't write. It can write a presentation, but I can't write a great one.
Not concern is that catalog when when a potential attendee looks and sees the same sets of words over and over get every session, they're going to get the creeps right, Like, that's not good for business. So you definitely have to push on. Hey, I like your top I like you. I want you to speak. This is the topic you're talking on. But you use chet GPT to write this thing,
so it looks like every other abstract. Like you have to write a better prompt or at least edit this into something that looks like it's you rather than a piece of software.
Yeah, so especially here's here's a tell if the first three words are did you.
Know they I've got a lot of in the in this fast based movie based technological world in a world right the clerk's in opening. Yeah.
Yeah, we've started having to use because info has article submissions and we feed it through was this AI generated? Like my son's in college and all through high school and everything else, they've had plagiarism checkers and now they have AI checkers. Wow, LLLM checkers, ELM checkers.
I'm doing it. I'm doing my best to discourage people from using AI because AI just tells me you don't know what it is.
That's the problem. It's such an easy thing to say.
Yeah, I'm going to start using NS, which is natural stupidity because it's kind of the same thing, just in reverse.
But I mean again, I go back to it has value and when it's wrong, that's a feature, not a bug.
Yeah. Well, now I've been pressing against folks. Is like when you say AI, what do you actually mean? Can you articulate it? Yeah, so that it becomes a slower term, a term that needs to be qualified. Right, Why don't you use the qualified term? Then maybe we can get going, And I think otherwise we're all talking magic, right, yeah, right, if you just wait, AI equals magic. Sorry, no magic allowed. Oh you're using a large language. Well well that's not magic. Fun.
Yeah, And that's what the architectural intelligence is is figuring out when to use those actual AI elements. What are the real world things we can do? And that just comes down to good traditional tradeoff analysis. We make trade offs. Is this the right thing or the wrong thing? Oh, it doesn't fit here. For all these decisions, I can put it in my ADR and say I considered an LM and we decided to go with a traditional write the code approach. But you've got to get past the
hype of hey, AI can do everything. Well, AI doesn't actually mean anything. It's just a marketing term. What is the tool and how can you use that tool?
And you know the person doesn't actually know what they're saying when they say that, because when you ask a question like that, they really tails ben yep, well you know, you know, sorry, we don't have Jarvis. Jarvis isn't the thing. So what do you actually got here?
Thomas? Is there anything we missed that you want to mention?
I just wrap up. So I started with the quote from Arthur C. Clark. I think it's his third law. The thing I want to wrap up with is his second law that the only way of discovering the limits of the possible is to venture a little ways past them into the impossible. I think there's a lot of this hype around AI and LMS and what can they do? But we don't push the boundaries, we won't find what
those limits are. So sometimes you have to basically believe the hype and go into that impossible and then we'll figure out where we connection get too.
That's good, Well, thanks, Thomas.
Has been enlightening to say the least, and it's always good to talk to you, so thanks again.
Always great to say you guys.
All right, we'll see you next time on dot neat.
Rocks dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two.
Thousand and two.
And make sure you check out our sponsors. They keep us in business. Now go write some code, See you next time.
You got jam, Vans
And