How'd you like to listen to dot net rocks with no ads?
Easy?
Become a patron For just five dollars a month, you get access to a private RSS feed where all the shows have no ads. Twenty dollars a month will get you that and a special dot net Rocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey guess what, it's dot net rocks from Bill. This is the last show we were recording. I'm Carl Franklin. We're here with Sef Juarez and we'll be talking to.
Get that out of there. Get a little fam Sorry, you gotta get it out. You know, we're a little punchy. This is not only the last show of the day, but it's the last show up. Oh, if you got a punchy a show with a punchy dude, it's gonna be great.
It's gonna be You've been working hard, that's right. I just want to start talking to Seth. Do we really have to do the intro stuff? You should do the interesting?
We gotta do it. You got to pay so you know it's episode nineteen fifty seven. Wow, that's the year I was born. It's not true. It could be in a different universe very close. It could be in a parallel universe. If you know quantum theory, the many worlds as you know, Richard In posits that perhaps there's another universe where I was born in nineteenfty seven. There's a million universes. That happened, sixty seven year old set. I just wanted to use the word posits in the thing.
And so now my job is I'm done. You must have a master's degree.
I have something. So do you want to do the history first? You want to do I didn't do the history. So what happened in nineteen fifty spot Nick one?
Okay, the first the first artificial salarate over the world, and then sput Nick two. They put a dog in it and she died. No, actually no, she was in the Guardians of the Galaxies. Right, wait, that's not real. We want the historical documents and it's worse than any think. She didn't she didn't die and re entry, she didn't suffocate. She overheated. There wasn't temperature controls oh thanks.
Sometime around here in the nineteen late nineteen fifties, electric boat from General Dynamics was established in the Nautilus.
That was no electric boat? Is that? Is that a like a line dance? What is fifty five? Isn't it? The electric boat? Is that like the electric slide? But no, no, no waste? The nuclear submarine? Oh yeah, the first nuclear summer, theatlis.
That's right, Okay, I'm sorry. I had a forward memory of.
I also usually referenced a computer, and in nineteen fifty seven, IBM released a compact computer, the six', ten only eight hundred, pounds much smaller than the previous. Ones it really was an improvement there you. GO i, mean if you want to get, fit you get one of those commonract eight hundred pounds. Computers also released the first compiler for. Fortrend hey, yo, wow still running some code out. There there's as for
trend dot hold. On So i'm not. Kidding there are some linear algebra operations that were written In fortrand that people were, like let's just not do that. AGAIN i think it's like some WEIRD langsos method for, computing like singular value, decomposition that they're, like let's just not do that. Again somebody did, that and we're, gonna we're, gonna we're gonna honor them and we're just gonna leave that. Alone. Wow. WOW i actually contributed this.
SOMEHOW i actually remember taking A fortran. Class but it was like in the. Eighties, yeah a little further, on yeah, too it was like in THE i did fourtran.
Three oh is there a sound? Effect weing? At BUT i love? It all? Right, well, anyway anything else in nineteen fifty there was a lot Of there was a lot of. Things there's new clear. Test it's all kinds of horrible stuff listed with, computers all, right and speaking of atomic, fires what are we talking about? Today, well we gotta finished the fish such a. Hurry i'm not in a.
Hurry, okay all, right let's do the better note of framework roll the, music all. Right, so as you, Know Jeff fritz AND I Jeff frits For microsoft And big INTO ai And. Blazer we do This blazer puzzle.
Show AND i was just.
Talking to him about all of the cool THINGS i saw at the, keynote including, you my friend oh hey uh at the keynote it build and of course he's hipped to all of this, stuff and we were kind of talking about how and we talked about this with some of our guests this. Week about how the prompts are becoming code and instead of having code, libraries people are going to start sharing prompt. Libraries Prompt so what
Did fritz point me? To he pointed me to sharp, site which is all these prompts for Doing blazer applications with the.
Agent oh, interesting do they? Work?
Yeah in good hub co pilot so code style and, structure naming, conventions dot net specific, guidelines sho air handling and, validation like all of these great prompts to help you create Great blazer.
Applications that's all the. Agent how about? THAT i think that's. Great anything that we can do to further the work OF, ai because you, Know i've BEEN i was like on show like, SEVENTEEN i don't, know something like that eighteen AND i was, like machine learning is the future AND i was like What and we didn't understand. It then we still, don't but at least it's. Working but it's, true and you just still understand. It we just, believe you, KNOW i really excited that people are, like let's use
this stuff right five seventy Six july of twenty. Ten oh my, gosh aires on machines in eighteen years AND i just get more. Beautiful i'm glad this is. True in this, medium it's really hard to, know but it's for. SURE i want you to imagine, it all.
Right richard Save, us who's talking to us, today.
Grabbed to comment off a show nineteen, nineteen which you did in twenty twenty, four a little more recent In october With Prascent boy here he's AN mvp DOING ai related stuff that he wanted to talk, about and we did talk About Copilot, studio WHICH i believe has now been renamed because. Reasons, yeah And John tallon had this great comments from a few months. Ago he, SAID i enjoyed the. Show chat BASED ai apps are quick to stand, up but tough to get right every time a man
rag dazzles that. First but yet it's limited without deep. Engineering. Ooh in real, conversations not everything is a, question and bought led chats are no. Different we must parse the user's, intent evaluate the quality of their, input then apply a chain of erication to fact checkbox. Responses i'm a practice director in data AND II, ai And i've worked in Gen, Ai AZURE Ai studio and Co Pilot, studio and despite the Marketing Copilot studio isn't for your Quote jim From.
Accounts it's for seasoned consultants Like brashant who know how to use the. Platform using it for complex, interactions like a rocket launcher on a. Fly, yeah it, works but it's a little over. Complicated rocket launcher on a. Fly it's, like use. It, yeah let me say that. Again using it for complex interactions is like using a rocket launcher on a.
Fly it's kind of taken a nod from my term swatting a fly with a.
Buick we had a term swatting a fly with a, shotgun and my algorithms professor was, like we have an algorithm for sorting that's faster than en log, in but only with giant data. Sets it's like swatting a fly with a. Shotgun, yes, yeah. Nice we should only explose enough complexity to get the job. Done letting use His harner's AI's potential without feeling. Overwhelmed after, all the goal is to DEMOCRATIZE, ai not. Intimidate what year was? That
that was last? Week, yeah this is like a few months. Ago the answer is you just, wait wait till next. Years you. Know one of the things that CAME i felt came out of build was this sense that nobody's dominating in the space at. All, yeah there's lots of different, models there's lots of different. Tools the fact that everybody jumped ON mcp is. Clear nobody has the confidence to, go, oh we'll go it. Alone we're all trying to find standards in inter interapt with each. Other. Yeah BUT i
mean even EVEN mcp has. Limitations AND i have thoughts on, that And i'm tired And i'm just gonna say WHAT i. Think Oh, NO mtcp definitely has some. Issues there's no two ways about. That but let me wrap up With john. Here, john thanks you so much for your, comment and the copy of Music coba is on its way to. You and if you'd like a copy Of Music, COBE i write a comment on the website at dot NetRocks dot
com or on the. Facebooks we publish every show, there and if you comment there and every read it on the, show we'll send you a copy Of Music. Koba And music.
To Code by is still going. Strong twenty two tracks to help you get in the zone and focus twenty five minutes each to coincide with The pomodor, method and they will help.
YOU i actually bought the first. Ones yeah a long times years.
Ago, yeah it was only three AND i did A, KICKSTARTER i did.
THREE i got the. THREE i still have THE mp three is somewhere in my junk drawer of one.
Drive, well now you can get the whole collection IN mp three flak en. Wave so you can go to music To code by dot net for that or you, know you could just send us a comment or leave a.
Comment, yeah, absolutely and reading on the show you'll get a copy of MUSIC o.
Bye, yeah all, Right So i'm going to introduce.
You for real areas here we go That i'm introducing.
It, yeah that voice that you've been listening to the can who it is Is Seph wares and he works in THE ai platform group At. Microsoft he received his bachelor's. Degree that's, it that's your. Title Now i'm going to talk about your education because that's the part of your. Bio we see his bachelors and computer science at you n L v saying place that Guy fieri got his.
Degree we were. Partners he's a he's a machine learning master.
Absolutely with a minor in mathematics and complete his master's great at The university Of utah and the field of computer science And. SETH i have a picture that BECAUSE i was in the front row at the build AND i took a picture of.
You oh what WAS i? Doing? Well you you opened the.
Show you came out and, SAID i Am seth and you, know with with somebody, else AND i DON'T i didn't recognize. Her BUT i took a picture of you two and here it. Is oh my, gosh That's. Kadasha, yeah she is delightful human. Beings SO i sent this To, kelly my, wife you, know, yes of course she actually met you BEFORE i met.
You, yes she did, so and, she.
You, know she says back to me and she, says that's a great. Picture but who is that black lady With Ricky?
Martin but, WOW i do get more. Beautiful AND i, said, again this is the medium for me to say these kinds of. THINGS i am getting more. Beautiful look at the.
Picture your name is not on the, screen it's it's her. Last, yeah so she wouldn't know and she didn't recognize.
You that is so. Funny then WHEN.
I told, her she, says, oh, yes that is.
YEAH i actually owe her a. Call she called me the other day AND i haven't called. Back, oh, okay she's, delightful she. Called she checks up on. Me she does. That, yeah, yeah so. Nice well you do need. Supervision it is. True my wife has had me now for twenty two ish, years and so during THE covid she may be. Tired during THE.
Covid, times she reached out to friends of mine and hers that she hadn't talked to in years and, Said, hey you know, What i'm just calling BECAUSE i want to check.
In it was.
GREAT i don't want, anything you, KNOW i just wanted to, Say, Hi.
GOSH i wish the niceness would rub off on other. People, yeah it's not working. Right all.
Right so we're talking about agentic, hey AND i got to tell, You i'm blown away by WHAT i saw. HERE i, MEAN i know that there have been other agents and products that work with agency and all that, stuff and the guys From manet have been into, it, right but they all said the same. Thing it's, Like, Wow microsoft is kind of catching up to what these guys have been. Doing AND i hope they dominate because they have the, power, Right microsoft has the power Of
azure and all of these. Things and WHEN i learned ABOUT, mcp we've been talking about anybody who's listened to the last six, shows oh, yeah knows about this about the, okay is it.
Model model context context.
Protocol so this is the protocol by which agents can talk to your resources to do things like talk to YOUR sql, server talk To azure resources and all that. Stuff and you can just tell it to go do, things, yes and it will figure them.
Out.
Yes the combination of AN llm like get Hub copilot AND mcp connections to other resources just blows my. Mind it changes the.
Game, Yeah i'd love to get more low level, though so folks have an understanding of what an agent actually is because there is This what happened was and you were there with me for fifteen. Years i've been in this space for fifteen. Years nobody, cared and then we flip from nobody cared to this stuff is like jungle, magic or this stuff is like going to throw you into outer space or.
So we didn't have to create the. Models, yeah that was the hard.
Thing it's. True but but what happened is we kind of missed the in between part Of, hey let's explain what these things are doing and why it's actually. Useful because there's a THERE'S i don't want. To as you, know we had SEVERAL ai winters because of the over, hype AND i AM i am concerned that if we overhype too much that there's going to be a correction when there really doesn't need to be because it actually this time does do qualitatively cool. Stuff, yes and that's
What i'm worried. ABOUT i, mean the winters were mostly induced by the fact that there was a limited utility what was being made, exactly and there was a limited of computational power to actually make the model the thing we wanted them. To hinton's famous papers actually the paper talking about backpropagation SAYING i can't do with this equipment in like nineteen eighty four mm. Hmmm and then In suskovar does off that paper in two thousand and nine
and they start working on the image net. Problem, yes and those models are quite simple as compared to the ones that are now FOR. LMS i, MEAN i can get into the architecture and stuff of. It they're just much. Bigger and the thing about the thing about neural networks of that size Is i've spoken To Turing award winners about, like how do you shape these actual? Models to do the things you, Want and the reality is we don't
really have a good understanding or. Explanation but what happens is they're innovations that come out like a transformer model with attention that was really designed to do translation that, somehow when you made it, bigger learned language, well presented language in a form that you could anthropomorphize. Correct and
that's that's the Part richard does not like at. All the thing about it is is WHEN i picture these things in my, head it's like the matrix for, me Where i'm picturing like there's, weights multiplying vectors and then going over some kind of smoothing function to, remove you, know the linearities so that these scenes can predict complex.
Things but in the, end all these things are doing is you're getting a string of words that are broken up into vectors that get passed in and multiplied by the transformer model to figure out how these things are, related pushed through a giant neural network multiple, times and the output is a giant vector of the entire space with a probably distribution that says which token should you put out? Next so when it looks like it's, typing it isn't it's just going back through the, model and
that's what these things are. Doing and as, programmers the only thing we can control is what goes in the front the. Prompt and that's the beefy, bit right that people don't. Understand at that, point when we first started doing this, stuff it was magical for. Chat, Yeah but then early, on about maybe like six months into, it folks were starting to use these things for something that
we liked to THAT i like to call intent. Mapping Cassie brevy AND i worked together for a couple of years and that's what we called, it where we would make THE lm like look at a block of text and, say is this question a support or a sales? Question and it was at that point that these, things because we allowed them to decide is it a support or a sales? Question when the code, branched those things all of a sudden were imbued with this notion OF i
can make a CHOICE lm on what this thing. Is and at that point those things became, agents, yes and and and because we trusted them to answer our. Questions. Yeah. Yeah but but but nottice that WHEN i was doing this, ENGINEERING i narrowed the task to very specific you will say either support or, sales and that's. It and what that did is it allowed me to create a prompt that made it so that any user input that we put in, it it would tell me support or sales every.
Time and my code had an if statement if support do, this if sales do, this, right and that at that, point that is the most simplistic way of thinking about an. Agent when AN lm has the has the ability to choose reason overset of data and then act on the reasoning you've. Generated and effectively that became a squishy programming control structure and that's what an agent is. Fundamentally but you're, like but it's got to be more than. That seth
what about the tools? Stuff at that people are, like, well let's do structured output so that it could output adjacent. Thing and if we can output adjacent, thing why can't we output adjacent Thing that says this function with these. Parameters and at that point those things became tools and now instead of supporter, sales it, said run this function with these. Parameters and that is the extent of all of the, magic sauce of all of anything.
Agentic NOW i want to say something Here, seth you are among some of the most brilliant people That i've ever.
Met, oh thank.
You richard is one of. Them you are one of. Them but the other people are like dev express. People, Yeah Mark miller And Richard morris like these guys and. You, NOW i want to tell the listeners if you don't know Who seth. Is, seth you had an open SOURCE AI mL library Before microsoft hired. You, yeah before they Had AZURE. Ai you had this open source. Library we did a thing about it on D NR tv that showed it to.
ME i didn't understand. It, yeah of COURSE i kind of. Did but but you were doing.
That you've been doing this longer Than microsoft has been doing.
It way before it was. COOL i was doing before it was. Cool the secret, sauce, though is machine learning has been happening since nineteen fifty. Six Frank rosenblatt The, perceptron which is the basic foundation OF.
Ski so that was the last, show righteen Fifty.
Oh, hey, yeah let's call back To Yesterye, frank we've missed. Now but the reality is That microsoft's been DOING ai in the form of machine learning for a very long. Time WHAT i was doing IS i was trying to bring it to the dot net community BECAUSE i recognize that there is a space for algorithms which are learned instead of, programmed AND i thought that IF i could do a. Thing OBVIOUSLY i haven't kept up on that for a while because it was just. Me BUT i
implement a whole set of stuff to make that. Easier but my goal has been for the last you, know fifteen twenty, YEARS i want people to know exactly what these things are, doing so that with, confidence you can make them do even cooler.
Things, right, yeah, Yeah and so now you're At, microsoft you're working on THE ai.
Stuff that's The azure. Stuff that's. Right The Italians channel, Nine oh, yeah, No channel nine was. Great you had such a good. TIME i really. GOOD i love. That and then somebody figured out actually knew them all, stuff and it was. Important it's, like you come with, me you,
know CAN i be? HONEST i ACTUALLY I actually it was different than that because AS i started doing The channel nine, stuff the reason WHY i knew what questions asked is BECAUSE i already knew their content AND i knew how to make them talk about and make things. Sing but people just started saying stuff to me, Like, seth don't you ever get tired of all this technical? Stuff And i'm, like oh, man MAYBE i should move
into something more. Technical maybe. Maybe so THEN i moved into DevRel FOR aiml and Then John, montgomery who was A cvp Of young he was, Like, seth come work for me in THE ai core.
Group so that's what you obviously knew your, pedigree, right they knew they.
Did nobody, knew LIKE i THINK i think people record like see me and like Because i'm pretty jovial and and BUT I i.
Think you don't come across as AN ai.
Scientist, NO i KNOW i do. Not and but if you looked at, me obviously we've said this many, Times i'm very beautiful in, appearance and people would not Like mark. Exactly people wouldn't, know BUT I i really have been doing machine learning. Forever and, again my primary goal is for people to understand at a profound level what these things. Are and to, me they are basically a new programming control structure which can Take english and point you to
what to run. Next, yeah fairly still Constrained, english, yes, well, no even Regular english with AN, lm because if you can you can type into AN ll m like Bad. English and for, EXAMPLE i did a demo at my talk two days ago WHERE i showed him the supporter sales thing AND i, said my tent is, broken, yo AND i spelled it wrong and it said support AND i was, like what's the borst tent in the, world? Yo and it was like burst and not b e r S t BECAUSE i fat fingered it. Right and
then it was, like, uh. Sales notice that because it's a large language, model it's able to reason over the text and then create the output that we. Want and if that output is a, function then it tells you what to run. Next so it.
Didn't understand, bursts so it figured that you really meant worst.
A best, Best, yeah, okay best b e r S. T and it was, like even if it was, worst it's still a sales and it still was able to figure it. Out and and that was with me, again and THEN i. CAN'T i don't know how to emphasize this is. Enough, basically you get a control structure that gives You english and tells you what things run. Next WHAT mcp does and whether it does it well or not is neither here nor. THERE i don't think it's Necessary what it does is it allows you to query
what functions can you? Call and what it does is it maps those into the tool calls that are available to the L, m and then it can call. It but you can do that With swagger AND. Rest, sure, yeah but it's.
Not it doesn't have the the almost it does.
Reasoning it, does but it does because the reasoning happens in THE. Lm and what it outputs is it outputs what functions Are swagger AND rest don't have that they, don't but you don't need it BECAUSE, mcp if you look at, it is basically Like. Swagger it tells you what tools are, available and it's LIKE, rest and it tells you how to call. Them, Right and the thing THAT i like about like those kind of and OPEN, api which has been NOW i think the Next swagger is, that,
RIGHT i don't. Know but with OPEN api and WITH rest you're able to do the same things but with the benefit of like twenty years of security, handling with, headers WITH, ooth with token, negotiation et. Cetera and so that's, why like IF mcp gets you, going like WHEN i USE vb, six, right it got me. GOING i loved it because it was able to. Get but then when you were, LIKE i need to be more memory, CONSCIOUS i needed to be more. Careful then you can skip
down a layer if you if you want. To but, AGAIN i love THAT mcp has gotten people thinking about. It but the reasoning doesn't happen WITH. Mcp, no it's, yeah it just tells you what function are. Next but the magic that makes it look good is once it runs the, function there's a thread that has like what the QUESTION i? Asked it asks for a Function i'm
like pointing like people can. See and then when it executes, out it puts that function return call back on the thread and THE lm is called again with that on its, thread and then it comes with the. Answer and so that actually going out from THE ai space is effectively bridging the gap FROM ai reasoning to perative or functional. Code. Right how we MAKE sky? IT i get? It, NO i mean it's NOT i don't even think IT'S.
Sky and it may not be the best security wise right, now but it's clearly where everybody wants to.
Go, yeah it's because it simplifies. It but my sense is that as you see, This i'm the prognostica, here, right BECAUSE i like doing. That as you see this thing, maturing it's going to be indistinguishable from rest right and discovery. Protocols that it's the way the embracement OF mcp says to, me everybody wants to interrupt with everybody. ELSE i fact that we could come up with a better way to do, it and in fact already. Have we could just be
publishing APIs for crying. Out, Yeah And i'm telling, you like we we had a phase At microsoft and bless Our hearts where we went full. Wisdole, yes and.
That's what The southern people say about the town drunk bless his.
Heart, yeah bless our. Heart But wisdole was the same, thing the same kind of desire to to abstract over. Primitives and then we went back to primitive still being WSD l Or Web Service Discovery language. Wisdom does that sound? Familiar, friends done done? Dumb and so What i'm telling you is that and some of the things you saw with N a Web the reason why this is such a cool.
Space if we haven't talked about AN a web on the, show oh, really yeah this WEEK i haven't.
EITHER i started looking at. It but the intuition here is if you can language do stuff and then call stuff and then language do stuff on the, Out like what you if you hook this up to the in language tool, stuff, Right you're? Fired, yeah you're my problem WITH nl. WEBS i keep, thinking why are you talking About?
Dutch people's confused The dutch. Web but hopefully that helps BECAUSE i really want to emphasize that once you know that that's what it's doing at a low level and you play with it that, way it changes the way you think about These. SURE i had a journalist talk to me and, say, Hey, SETH i get a lot of questions about SHOULD i trust these? Things and my response surprised. HIM i, SAID i don't trust these. Things, no don't be. Silly that's WHY i program them the
WAY i. DO i restrict what they have to. DO i restrict the tool calls that they, have and then when the tool calls are, CALLED i write the code that the tool that what the tool responds. With because all these things are effectively becoming part of the prompt for the next tokens to. Generate and that is when you get that, Intuition you're, like, oh that MEANS i can do this ANYWAY i want to ye.
And that's that's WHERE i think we've been talking about this this week, too that the attributes of the program or the future are going to be imaginations over technical probsolutely imagination mean and we may be held back a little bit by what we think we can do and what we can't. Do but somebody who doesn't have that kind of you, know, limitation self, limitation will be able to do amazing.
Things. Yeah like if you watch the keynote on day, two you Saw Amanda foster And elijah on stage talking to this thing that it was executing a bunch of stuff on their. BEHALF i think it was. Software it was literally software BECAUSE i KNOW i, WROTE i wrote THE i wrote the website and the back. End amanda worked on the, agent And elijah worked on the on the. Evaluations but the thing about it is it was actually running.
Live that thing was actually happening. Live and one of the things that we had is once you get into a single, agent it makes, sense but when you get into multi, agent you wonder, like, well what is the interface between one agent and? Another and the surprising thing is that the interface is actually language, weird which is, weird, Right so for, Example i'll give you, example the voice agent that we, made and if you watch the, keynote
you'll see her talking to. It the voice agent is using THE gpt four oh Real TIME, api which has the also has the ability to issue function. Calls, Okay and so what we did is we collected the list of azure Ai foundry agents that we made Andree foundry AND i just loaded them up and then she picked the ones she wanted because she was working on. Them and what it did is it mapped those other agents as tool calls interesting to the voice. Agent but you're, like,
well what did you pass to the? Agent, well the agents themselves when you run, them have this property called additional, instructions which means you can augment their system prompt and then that's. It that's all you can. Do so WHAT i did IS i mapped all of those agents as tool, calls gave it a function. Name you're back to sales and support, again that's all this, IS i. Know and THEN i, said you passed voice agent additional instructions and
ask it for what you wanted to. Do so the voice agent then is taking the initial requests and assessing it to go which agent to go, to back to your sales and. Support that's. Right then taking the portion of the requests that makes sense for that, agent plus the additional, instruction let's send it on. Down but it's even more than. That it actually asks, it can you
help me with? This and we were me And amanda because it's like in the two in the, morning we're looking at this, stuff like because we're on this stuff all. Night we looked at the law because we Ad jason printed out everything and we could, see, uh image agent system, instructions you are to generate a blah blah blah. Question can you generate an? Image it was Actual english and we were, like holy, cow this is this is amazing and the software being polite to. Software, yes it.
Was really cook but but it's nothing. New if you look at the S MT smtp protocol when you connect When i'm When gmail connects to an S mvp, server it, says you, know, hello And I'm Carl, Franklin carl frank and THE smtp server actually writes, back, Hello Carl, franklin nice to meet.
You, yes but we're not talking part of the. Protocol it's Not eliza in This, NO i, KNOW.
I, know but but you know that those kind of friendly things have been.
Protocols for a long. Time but the difference is that the only things are being generated by the through the agency of AN. LLM i get. It it's, big big. CHANGE i do question if if if the politeness affects the prompt. Results, yes it absolutely. Does And i'll tell. You i'll tell you what happens And i'll tell. You so there was a part of the demo Where amanda generates an image for a hackathon and they AND i don't think they understood because someone backstage is, like, hey
can you change the image to something more? Happy and we're, like, no it generates it every. Time it's it's doing it on its. Own and a man is, like but we can change it by telling, it please make the image more, happy and like we made that change in The english and the and not in the voice controller because that's the thing we talked, to but in the literal little agent the, customer and it called, it called. It and then it was, like and now everyone's happy at the.
Hackathon and and because everyone thought we were using some like pre baked image and, no it's not every, time and then we did, that and then it was really cool.
All, RIGHT i think we got to take a, break so we will be right back after these very important messages don't go. Away did you know there's a dot net on aws. Community follow the social media, blogs YouTube influencers and open source projects and add your own. Voice get plugged into the dot net on aws community at aws Dot amazon dot, com slash dot, net.
And we're.
Back it's dot net. Rocks I'm Carl, Franklin I'm Richard, campbell And Seth warez is. Here we are JUST i feel like we could be talking for, hours hours an, hour so much to talk, about but we're talking about agentic coding and. PROGRAMMING i Took microsoft Agent mode In Visual studio for a spin yesterday to go.
Good, okay it went really.
WELL i gave it some tasked to create some stuff THAT i knew how to do BECAUSE i wanted to. See but there was one very tricky thing that it was intuitive for it to, do but it didn't. Work so it's not like agent mode is going to compile your code.
And see if it.
Works it's just going to do what it thinks is. Right, yes, SORRY i didn't meet us the ord. Thing it's going to use what it assesses to be the correct. Answer so but WHEN i went and told, it, no that's. Wrong this doesn't work on this particular. Object you're going to have to subclass, this it, said oh. Okay AND i didn't tell it what to. Do ALL i had to say was you're going to have to subclass this, subject and it figured it.
Out it's pretty cool. Stuff one of the things that you're going to find is that if you ever have an agentic thing that is not quite doing what you want in, general from an engineering, perceptive for, me it means that they've overstuffed what they want a single agent to. Do so the actual engineering problem becomes agent specificity versus agent. Count and that's an engineering, issue, right because if you have too many, agents it might cost, more or it
might it might work. Slower if you have too few, agents the recall isn't going to be as, good, Right and so the actual engineering problem is that it turns out the agent you use was. Able once you specify down a little, bit it made things. Better but if you think you're going to make a single agent with like a two page prompt with fifteen, tools it's not
going to do the right. Thing and that's the trade off from an engineering pler because now that you know you have to make it, specific you have to limit its tool because it's an interesting switch from what build up lms the first, place which is making them bigger and. Bigger now we're, going, hey if you wanted me to respond. Better but see that's the thing, small that's a. Thing it's not THE lms that are getting, right it's the tasks that you're assigning THE. Lm so it's THE. Lm
let's just fix THE lm for. One let's just SAY gbd. Four, oh like that models, Huge but like you are gonna narrow the agent's task to your job is to send an email to the right. Place and then here are the tools send email or delete email or. Whatever those are the only. Things that thing is gonna be very good at sending, emails and it's gonna be very good to be, like, HEY i need to send an emails to whom please type or tell me the. Email and
then it's gonna be, like please validate the. Email what content do you? Want because it's trying to fill out the parametery of the function call because that's what the prompt wants it to. Do and that's what's. Happening so for your, case LIKE i worry like even in some of the tools that we, make maybe At, microsoft maybe they're too broad and that's where you find this notion of. Hallucination now people aren't gonna like, this but the elem's job is to actually make up. Tokens if you give
it too broad a, thing it's gonna make stuff. Up if you get it to if you give it a narrow, thing it's gonna make the right stuff. Up BUT i thought that's.
What the point of the agent THAT i access in visual studio, is and it's NOT gpd four oh with this broad uh set of knowledge or. Whatever i'm, Sorry, RICHARD i didn't mean to use the. KNOWLEDGE i have to apologize every TIME i answerpomorphize than LL.
M but he is. Right thank you keep him on. Track, yeah, No i'm. SORRY i apologize now ALREADY i. Apologize please be with the pigmies In New. Guinea. Yeah.
Amen but it narrows the focus to truss the things that are in. Code so theoretically it should know everything about the Language i'm using the, markup you, know If i'm Using blazer or. Whatever so it's got to UNDERSTAND html and JavaScript and all Those you're.
Gonna love this. Answer it knows. Nothing it knows only what You i'm trained on. It, well, yeah so there there is a, yeah so that's WHAT i. Mean, Yeah so there's certain models that are actually Like, codex for, example is perfectly trained on, code but it doesn't know like in my, head it doesn't know the. Languages it knows that if you give it this sequence of programming or, questions it's going to return this. Sequence let me just add this to the.
EQUATION i did the same problem with chatchipt AND i did get the right. Answer, yes, so but the thing that was The Microsoft agent mode In Visual studio did not give me the right answer.
Because it's a different.
Model, Right but wouldn't it be narrowed down to just the programming stuff like why did CHAT gpt with vast programming and learning that it did on all this data provide a better answer.
Because models are stochastic, processes and the thing about one model versus another is one will do better with the same prompt or. Not and so that's when he gets the college. Words that's five dollars five dollar. Word it's a stochastic process because the output vector gives you the
probability distribution and samples according to that probably. Distribution so some, models depending on let's just say that use clawed in that, one or they are going to do different things and it all so the promptly makes a, difference and the tools that it has really makes a. Difference and that's where it sounds like an excuse to. Me it does and here's what they should have. Done just like we have unit tests for, code we should have quote unit
tests for. Agents and those things are called. Evaluations and so An azree foundery we have a bunch of evaluations for, example is it follow the right? Task is it? Coherent is it? Relevant is it? Grounded we also have these other agents whose job is to attack our. Agents they're called the red teaming, agent and its job is to literally send garbage at it to see how it. Does
and so that is the unit. Test and the difference between a unit tests and an evaluation is that an evaluation you need to give it enough examples to be satisfied that you're going to be. Okay it needs to be probabilistically, relevant, Yes and then once you're okay with, it when you put it in, production you monitor it with the same evails and send alerts when it goes.
Off the rails diverts because when you're looking, at for, example regulated, industries regular, industries like if you make a, mistake you got to report, It but they also want to know how do you fix these? Things and that's probably more, important and that's the part of regulated. Industries when you use this s uff you can see where it went wrong and you fix it and then you go.
Back SO i think the real answer, is, hey we just launched this, shit give us some time.
Time i'm not part of that. Team they mean they could have made it, better.
Maybe BUT i mean it literally just came, out like you, know so and.
They'll learn a lot, because, like for, example the question you asked it maybe something they never. Anticipated that's, Right, yes so there's Learn so there's some alert that probably fired during THE evls that's, saying, Oh carl did not get this thing that he, wanted and so they update the prompt or a new tools AND i expect.
That. Yeah, so BUT i love the, idea and you, know talking To Scott hunter and some other, people the possibilities of things That i'm to do with it in the real world from my customers is going to blow their.
Minds, Yes and in terms of like the stuff you can, do it's the ceiling is very high because the biggest barrier to programming isn't like the new algorithms or the new it's like literally people using this stuff and having the like like someone Putting bob in the age. Field you, know that's the kind of dumb stuff that we write most of our code. For right ll ms together in an agentic, mode are going to soften that so that we don't have like why do we need a web form?
ANYMORE i can do like my name Is seth and you can use my current location to find the.
Address, now you're absolutely, right and YOU i is going to, change yeah from you. Know but but users are trained on, forms aren't. They, yes you're going to have to train a user to all. Right so here's here's a. Nightmarebox nightmare. Scenario a textbox to tell the application what you, want and somebody comes in and, says, hey you want A turkey. Club i'm going to The. Delhi, Yeah i'll have A turkey. Club and that's Writing turkey, club you know in the
in the. Thing you know WHAT i mean, like there's there's. You now you have to train the users as to what to say or what to, type and that's the.
Beauty. THOUGH i think given a properly built, agent that turkey sandwich is just gonna be noise to. It like it's gonna look for the things that will force the linear algebra and the calculus to produce the right next.
Time, yeah and then there's gotta be validation and. Verification did you really mean to say? This is this what you really?
Want you? Know and ALSO i think the software is fairly good at recognizing a contact shift and SAYING i don't THINK i was supposed to listen to, that, yeah.
Right and Also i'm assuming that you're speaking to it right when you really need to be, Typing like you don't Want bob From engineering to pop his head in and, say delete all records from, customers you, know.
DUDE i saw AN mcp server that was attached to a file, system AND i was, like maybe let's not do. That, yeah it seems like about on mind's. IDEA i remember when the you, know the First amazon devices showed, up and right away it was, like send my porn collection to my mother.
Joke oh my, gosh send one hundred pounds of cement To Richard.
Campbell oh my, gosh so. Dumb but, yes BUT i think LIKE i, Think LIKE i said. It it was surprising to me how the the controller agent in this case was communicating with other. Agents, yes that. Was and then you, Know i'm not surprised by this stuff BECAUSE i know how it. Works, Right but that was the part that was, Like, okay maybe there's something. Here, well and we get back to this whole like The battle Of VISIBLE. Ai, yeah, right and we still haven't really
fixed in there all that. Problem, no there's no way we can fix. That, well and that the Whole i've read the anthropic paper on the reasoning models and what's actually going on. There it's calling itself right couple. Times, well first it gets the, results and then it peaks pieces of the results and feeds it back into it to say why WOULD i have said? This so it literally is in reverse. Reasoning i've already decided what the answer, is but let me break it down into pieces so
it looks Like i'm thinking about. It, yeah that doesn't seem to be a good way to do, it but it makes people, happy, Right, well it doesn't make me, happy well until you know what's going. On and it's LIKE i just.
Read something in the news today that said THE ceo Of anthropic admitted that he has no idea how this stuff for but.
You, know maybe like in his, defense like there's certain things that CEOs do THAT i just don't want to know how to. Do, yeah you know What i'm, Saying like there's only there's only so MANY i don't know if this is, true but there's only so much space in my brain for things, nice, Right and it's LIKE i have, seen like BECAUSE i work with a lot of our EVPs and CEOs and and all that, stuff and like And i've worked And i've talked to CEOs and cvps and it's just like MAYBE i don't want
to do. That, Yeah and and the fact that he doesn't know exactly how it works is, okay focus on the right. Things. Yeah and the other thing that for, me LIKE i mathematically know how these things, work BUT i don't expect other people to just to. Me it's just LIKE i love looking at like the beauty of the mathematical structure of these things and trying to think about why does this actually?
Work but you know, WHAT i feel better knowing that as long AS sef.
Knows BUT i think this idea that that neural net is is essentially creating a jpeg of, language lossy compression of language one hundred. Percent that is exactly. Right but when we, say and the whole thing, is don't put any too many pictures in, It, yeah you're just gonna get a GREAT FUZD i. Know and then but but, then but, then because these things are really good at, sequences think of it as any. Sequence so we have models And andrea found with that do protein. Sequencing, yeah,
well deep. Fault and if you think about, it you're, like, well how could that? Work, well because again you're giving it a sequence of, things and it's telling you the next likely. Thing and then it's doing it in a loop right while not end of line token or end of thing. Token, right that's what it's doing and get next probable. Words. Yeah and the thing about it is people will talk about the, temperature whether it makes it
more creative or. Whatever it's not making it more. Creative it's basically squashing the output distribution so other things are more. Likely and that's all it's. Doing and SO i hear it's making it more. Creative, no it's. Not it's changing the hand of the output. Distribution it'll make it, faster but you'll get. Crap it's not even. Faster it's exactly
the same. Speed it's just the probability distribution is warped so that if, like for, sure it's this, one if temper is, one it's like the probably is what it it'll pick the, right it'll. Almost but if it's, different there's other things that become more equally. Likely and so it's chain of like this, token now this token it Becomes they say it's more, creative AND i hate. That why DO i hate? That? Richard it's more, exactly it's literally just changing the output.
Distigue but, okay So joe, programmer, though do you think they need to answer pomorphize this stuff in order to talk about it or think about it in a in a way that they can use it correctly because everybody falls into this. TRAP i mean we're, Humans.
Yeah like humans are funny. Creatures we will we will pick up a rock and draw a smiley face on, it and so. Happy it's such a happy. Rock it must have had a really good Dayne and now. Imagine but isn't THIS i WANT i have to think this is what makes humans? Humans first, off paradolia is a survival. Strategy the tendency to see, faces whether they're there or, not means you were the first one to, run and so the tiger probably didn't get. You so that's an evolved.
Trait absolutely, right it's. Paradolia but what if this whole practice event memorphizing is the mechanism that led to us being able to, strategize like how you have to think like the animal to hunt the animal successfully with your limited. RESOURCE i, See and so that tendency again becomes a competitive. Strategy you're more likely to survive because your hunting is.
Morphout not to mention, mythology, right, well in, mythos in religion and all of these, things it's all about. Anthropomorphizing, sure and not just.
Why is the sky making loud noises?
Shaming right As thor is sending lightning, thing that's because he's angry with.
US i feel like we got to go the final mile and somehow get into The roman spears because we all think About roman, period, RIGHT i don't, Know but, well actually that project the name for the project that you Saw amanda And elijah Do we called it, sustinnao which Is latin for, SUPPORT i support, nice, Okay and SO i told him that and let's lie it for. Sales way to close the loop that was that was. Amazing this is a quality. Show my job as co
host Of. Professionals the joke that's but, Yeah so again the, key the key insight is think of these things as things that take. Language and if you have a prompt that's narrow enough and a collection of tools that support the, task it will tell you what consistent. Results, yeah, yeah That's these are the dimensions of a tool set for a group of programmers to be successful with correct they just had to learn the dimensions of. It and the joke THAT i, said is it like a switch statement
and if statement but. Huge So i'm going to try to make it a thing and call it the giant swift. Statement it's the swift for those that are. Swifties swift all.
RIGHT i figured out The latin for sales is.
Isn't That italian full Of? Chicago oh my, gosh that's. Hilarious but, yeah and so that that inside couple with the ability to evaluate them as unit with the unit with the agentic. Unit tests should bring this all back to quality instead of the schlop that people put. Out sure they, are and THAT'S i mean the battle here is how do you measure? Quality, oh there's a way to measure? It, yeah BECAUSE i keep being told this THIS lm is better than THAT llm is, like how
how would you measure? That what makes that? True so what they do is they give it a bunch of, stuff like, like they feed it stuff and then they. Measure i'll give you there's a couple of. Emails so there's something Like blue And, rouge which is like how close is it to the right? Answer? Right those are supervised tests where you're, like you give it a thing and you're, like here's the OUTPUT i wanted to. Have we've talked about this.
Before, Yeah blue And rouges is like those like supervised and supervised.
Tests. Right but then there's one supervised tests and you're, like, well how do those? Work, Well i'll give you an. Example there's a there's a judge BASED evl called, groundedness and what groundedness. Measures it measures how rounded a response from AN lm is in the facts and context that you gave. It, Yeah and so for, example the three pairing that you give it for the unit test is sas,
asked how much does your you, know outdoor tent? Cost the context is we give it all of like the tent, information and the cost is in, there, right it's just like the slop of it and the outputs like the tent is ten. Dollars. Right so given given that a judge.
Base before the, tariffs now there are a lot, more.
That's, Right and now they're gonna be a lot. More but that particular three two, pole it's not, twopole but the three pole three. Yeah of the, question the context and the. Answer, yeah it's given to a judge to judge on a scale of one to. Five how grounded is? It and that judge is also AN Lllm, yeah okay another one, yeah and it. Works and that's why that's the whole idea of, Agency, yeah is that you have agents that are specialized in their different, things just like
you have you on your. Staff there special that's. Right the people of my staff Question, mark it's just, me all, Right.
But Joe developer who's listening is, thinking, oh this is. Great you, know after listening to these shows for the, last you, know the week that we've been doing, These i'm thinking now that you, Know i'm going to get rid Of bob and engineering And i'm going to replace no, agent And i'm going to set up this little staff of programmers under. Me agent programmers are going to help me do my.
JOB i don't think that's ever going to. Happen WHAT i think is going to happen AS i already. HAPPENED i think we're going to solve. WELL i mean those that think are they're going to solve it that way are going to come in Like i've seen vibe coders like put out stuff and be, like oh why is MY aws bill so high.
About vibe Coders and There's we've been arguing about whether that's pejorative or, not because the ryan it's.
Not, okay is it a pejorative?
Thing some people see, it some people see it, Goodness, Yeah But i'm talking about a professional co that maybe works by themselves and hires out other specialists to do that kind of. Stuff, interesting and NOW i can maybe spin off some agents to help me with that kind of stuff AND i not depend on them as.
MUCH i think that the agents are going to be able to do with a low powered brain work that we as humans do and and but the actual creative move forward and always human right and not and not be stuck in the drudgery of. That so you actually have more creative energy to do more experimentation. Exactly because then that's.
Why imagination is going to be the number one uh feature of the Rock star. Developers, oh there you, go there you. Go that's WHY i Think i've never been a rock star. Though, dude you are the more like a flute, player saying like You're Eddie eddie Van halen jazz PLAYER ai At Microsoft Cool.
Beings well look, again AND i just want to emphasize this is all accessible to. Everybody you can start playing with it. Today make it do a raw function call and see the output so you can see that What i'm saying is. True don't have it do the function, call just call it and tell it to do. Return do a tool call and you'll see it literally Returns. Jason that, says run this function with these, parameters.
And be as careful with your prompts with your agents as you are with your ten year old when they tell them to go to the store and get used xy.
Eggs or your programmer.
Husband, yes that's, right that's you, know the one about the. Programmer your husband and the wife, said go to the store and get a gallon of. Milk if they have, eggs get a. Dozen he came back with a dozen. Gas they had.
Eggs they had. Eggs but that's just how we have the whole world turned into paper. Clips but if you need to be, precise but if you think about, it that's exactly how you should think about these. Things that's. Right you have to be very because you want to. Listen you want to maximize the probability of it returning the right tokens based upon the ones you gave.
It so we're all in for a new world of discovery and refinement of search and refinement of. Prompts, yeah and the whole new.
World and if you go about retrieval augmented, generation it's literally the process of you get a, question you programmatically search for an, answer you put it into the, prompt and then you get the. Answer that's. It, yeah that's all it. Is that's How, yeah we just gave it a fancy. Name, Yeah WELL i don't like the TERM.
Rag it sounds it sounds kind Of but, YEAH i do a demo WHERE i do augmented, generation WHERE i literally go and copy From, wikipedia BUT i think the right answer is put it in AND i show it AND i was, like how does it know the right? Answer and it was, like, well because you put it. In, okay now you understand what augmented generation. Is the only process of retrieval was artisanal in my case BECAUSE i
went and did. It it was. Artisanal, yeah it's like a fine, cheese, Right but we's That and then if you think about we're too. Generation by putting the right context in and then coupled with tool, calling now you have a powerful. Combo and that's WHERE i, think if you think about it like, this you're really going to change the way things. Happen sure, WELL i also love how much you dilute the pretense of all this. Technology, OH i know it's the. PRETENSE i said that in
a very pretentious. WAY i. Know it's kind of.
Amazing, actually, yeah.
It was like a two fur three For, yeah there you. Go but like That, latin but it's a plural of. Slezolo. Right but if you think about, it this has been What i've been doing for the last twenty. Years, Sure, yes that's. True i've been trying to make this stuff lose all of its pretentiousness and when you, do it empowers you to do really. Things absolutely. Sure does the tools only get? Better that's, Right Brave New.
World and that's the last word from. BUILD i, Suppose WE'RE i, Love we're. Done it's TIME uh to hit the. Road thank you for. Listening thank, You.
Seth it's always a, pleasure always a.
Pleasure and we'll talk to you next time on dot net. Rocks dot Net rocks is brought to you By Franklin's net and produced By Pop, studios a full service, audio video and post production facility located physically In New, London, connecticut and of course in the cloud online at pwop dot. Com visit our website at D O T N E T R o c K s dot com FOR R S s, feeds, downloads mobile, apps, comments and access to the full archives going back to show number, one recorded
In september two thousand and. Two and make sure you check out our. Sponsors they keep us in. Business now go write some, code see you next. Time you Got Jent Middle vans