How'd you like to listen to dot NetRocks with no ads? Easy? Become a patron for just five dollars a month, you get access to a private RSS feed where all the shows have no ads. Twenty dollars a month will get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey Carl and Richard here with your twenty twenty four NDC schedule. We'll be at as many NDC conferences as possible this
year, and you should consider attending no matter what. The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Tickets at Cphdevfest dot com. Ndcporto is happening October fourteenth through the eighteenth. The early discount ends June fourteenth. Tickets at Ndcporto dot com. We'll see you there, we hope. Wow, you did it again. You found the last build show at dot net rocks. I'm Carl Franklin and I'm Richard Campbell, and Mark Brown's here.
We're going to talk to him very soon. But anything that you want to tell the people about our week here at Build you know, it's fun to be back in person. You know, everybody's getting better at conferences by bad, Like we're remembering how the conference. Yeah but yeah, I get Microsoft asked me to put together a podcasting space, so we brought in a bunch of other podcasters as well. Yeah. Then I figured, you know, we could make some dnrs while we were here. Yeah, and you know,
great line of guests, and this is the last of them. Yeah, the published wise, it's actually get a published in order, so it's all the last when we're recording. Yeah, third day, so it should be fun. Yeah, but I don't have any whiskey here, which is usually the last show of the day, the rituals. We're on the third. I'm a little bruised. Well you are, Yeah, I'm not. However, I've been a good boy. You're a responsible individual. I appreciate
it. I wouldn't say that. Let's not get crazy now, all right, roll the crazy music for better? No framework? Man, what do you got so better? No framework? For those who don't know, this is it started out as I would, you know, point my finger at a namespace in the framework. This you know, the any you know classes in dot net that we may or may not know about that kind of thing. But that got old real quick, and I started googling stuff and finding
eventually, do run out of framework? Do you kind of got through the framework? Yeah, it took a while, but you got there exactly. And nobody wants to hear, you know, a lesson on the bowels of the common language run time. Well, anyway, so Mark Brown is here, and Mark you can join in because I picked this toy for you. Yeah. Yeah, So who's a toy boy now right? I know I've become the toy boy, right. I know Mark is into a lot of things, you know, technology wise, but also electronic music, and I
know this about you. So I found I'm sharing a device that I have and I have a link for it on sweetwater dot com, which is where I bought it. Did not find it on Amazon, but they're out of stock. So this is going to be fun. When you know ten people who listen to dot net rocks, I'll call up and slam the back orders, slam the back lid. So this is a Kenton l N d R MIDI line driver. It's basically two boxes. It's a ballan. You know
what a ballan is? A way to use one transport for a different protocol, right, And there's a lot of Cat five, Cat six ballons where you just string a network cable between two things. You can do for video, for audio, for whatever. Well that's what this is. So MIDI Musical Instrument Digital interface has a problem that when the cables get over even close to twenty feet long, the latency gets high. It's a kind of serial protocol. It's a very yeah, it's a serial protocol, old school,
and so this solves the problem. It's always been a problem for me because in my studio, the piano is typically on the other side of the room from the workstation where I'm recording, because I want isolation and everything, and I've always had a problem with this, and so I've tried Bluetooth adapters and they're two Like, the latency is too much. This is the only thing that works. So I've basically hardwired this into my studio, one side on
the piano and the other side of my workstation. I'm taking a switch between either. It's litter just because Ethernet lends at a higher frequency. Yep, you could haliday a heckup a lot faster, that's right, okay, and maybe you did put a switch and you'd be fine. You're still talking milliseconds, yeah, exactly. So the piano that I have is the Yamaha Baby gran that has made built into it, and there's a Selenite under every key, so if you plug a MIDI keyboard into it and play, the keys
go up and down right. And it's also a great MIDI controller. So I just have it now hardwired in so at any time somebody wants to play the piano and record it. Let's say there's a drummer playing right and the piano playing at the same time, I can just have the piano player record MIDI and with no sound, and then edit it right to fix the mistakes, and then play it back through the piano and record it. Wow. So that's a trick that my clients love. Very clever, do that now?
Yeah. Anyway, it's one hundred and sixty nine bucks. And if you struggle like me, yeah for the pair. If you struggle like me with this problem, this is a solution. Whenever they get back and stock and ask for Mike God Love, he's my guy at Sweetwater. Yeah, and they have everybody orders a bunch of them then maybe they'll rush the back order back. Yeah. Maybe, yeah, yeah, maybe they'll offer me some free stuff. Who knows stuff is good. Free stuff is good.
Bought some stuff from Sweetwater. They're good people. They are all right, all right. Richard, who's talking to us today, grabbed a comment. Let's be clear, you found this comment, but yeah, grabbed a comment off of mast done. Yeah, it's only because we've already read all the COSMOSCB related kind of off of the of the website and so yeah, plus
other issues, but that's not important now. This comment comes from Hassen Savarn, who said it looks like the AI theme is replacing data topics and conferences. I get it. AI is everywhere lately. But AI depends on data. Yeah, you still need good data models, optimized databases, data quality. It all still matters. So organizers, please don't remove data topics from your upcoming events or net topics for that matter. Correct. Why would they
do that? I no, no, no, everybody thinks it's Python now with the AI think so on the other side is like his gigo ever meant more than it means right now, garbage in, garbage out right like that
was an old joke of computing decades ago. But what is a large language model but a garbage amplifier and garbage it spreads it everywhere, or a small language model for that, Yeah, hassan, great comment, Thanks so much, and I'll respond to you a masadon he send you a copy of music go by, And if you'd like a copy of music, go buy, write a comment on the website at dot at rocks dot com, or on
the facebooks or on masd on Twitter. All of those work. Well, read them all and if I read them on the show, I'll send you a copy of music. Go by. Yeah, blues guy, I'm a guy. Yeah, I heard they just got DMS on there. There you go. Well, you can find all the ways to get in touch with me anyway at Carl Franklin dot com, including all those things that Richard mentioned. So let's formally introduce Mark Brown. Been on the show several times,
but not in a while. He's been in the software industry for more than thirty years old guy, when I was younger, ripersnappers. Today he has worked for a wide array of companies, from ISPs to dot COM's building applications that ranged from the largest of back end systems to the smallest the devices and everything in between. Mark leads the growth team for Azure Cosmos DP. Welcome, Thank you very much. Great to be back. It's been a while
too long. And as you've you just caught me here and said, hey, we haven't talked about Cosmos debating around like you are correct, that is right. Are you doing any electronic music these days? You know it's been Uh no, I guess I'm just gonna say no, that's fine. Yeah, it's been hard to takes all it takes, you know, it does take time and dedication and yeah, and you've been busy, been very busy. Uh, life has not been the same. And I guess about eighteen
months since the AI detonation. Yeah, like chat GPT landed on my head like a ton of bricks right over that November twenty two. Yeah. Yeah, and I spent all of my Christmas vacation looking at this thing and thinking, hey, we should be there's there's a there's a there's a place for us in all of this. So it's your fault, well not all may some of it, because here it build it's all AI. I mean everything in the keynote is AI and everything everywhere is all averything. Yeah, well
you, let's face it, Microsoft has a great platform here. Yeah, yeah, jumped on right like I jumped in, and I started looking at l ms in this whole Jenai space and the technology here, and thought, you know, there's a place for Cosmos in building these types of applications, particularly when you look at things like chat. Like chat seems to be kind of the kind of the interface, if you will, a natural language interface
into that. And you know, customers have been using Cosmos to build just regular chat bots and chat applications for years, and that was kind of the angle I took. I'm like, what's our role in that? And things have been snowballing. In fact, it wasn't just me that was thinking and looking about this, but also our own engineering team and even the open Ai folks. Very soon after they initially launched, before that really exploded, they
migrated over from existing postcrists. They were running all of their chat GPT, all the chat history and everything on. There was running ont of some Postgrist servers and they were struggling as a numbers just a hundred million users in two months. Yeah, it doesn't just happen for free. It was overloading because they'd have to go and reshard their databases like constantly to try to keep up
with the growth in there. And we reached out to him and said, hey, I think we've got a way we can help you with this, and we walked them through Cosmos and explained, hey, this is you know, this is scale out architecture here right, so beause postcrists is still in RDBMS. So did they rearchitect their data? They did not really need to re architect. Migration was not wholly difficult, and they were able to do
it basically with the lights on. Wow for that. So they migrated over to us very early after they launched, and then we helped them ride from like a handful to I mean their peak was like over one hundred and eighty million daily active users. Wow. I think I remember the day that went into effect, because it went from you know, one word at time to just like yeah, yeah, And they did all of that with a single Cosmos instance. Wow. We just basically as users kept piling on and kept
basically adding chats and sessions to it. You know, Cosmos just ended up just just scaling out. That's what it does. It's a scale out architecture in there, so was that the largest customer that demand wise that they're not. They're not our largest customer, not even not even close actually, but they're I mean they're large, yeah, for sure. And I will say
this one thing that was unique was the rate of growth. For sure, the fastest app in history to reach those kinds of consecutive or daily acting. Us growing at that rate, I think it's it's pulled back from that one eighty, but now with four to zero out, it's it's spiking again, right, So they're they're likely going to go over that two hundred number, I'm sure at some point. Now. We've talked about AI a lot on this show since we've been a build and seeing all the great things coming in.
I've mentioned on more than one occasion that I'm so for developers embracing AI and for you know, especially the offerings that you have get hub co pilot and even the co pilot that is now set to allow us to sort of, you know, reach into our enterprise and see, you know, and ask questions in real language. That's great and and so, but I was
concerned on the first day. I'm looking at the keynotes and I'm saying, you know, there's somebody out there that's saying, you know, I'm going to use this to be my brain and you know, to make decisions for me. I'm just going to trust it. And of course you know that that's that's a problem. But then I was very excited to see on the second day the security layer that Microsoft ads over these large language models, so
that you know, that's something that chat GPT doesn't have. If I'm not mistaken, it's this thing that is looking for hallucinations and looking for inaccurate data and even malicious use of it. So important, and I was very glad to see that. You want to talk about that a little bit. That's the whole responsible AI thing you're talking about there. That's a huge focus and
emphasis on everything we do. Everything we produce and release internally at Microsoft, that's levernges of technology has to go through a pretty set of rigorous in seat of testing and controls and all that stuff to make sure that it's adhering to all those responsible AI principles that are within there. I'm really glad that we out of the gate are applying this to that, and not just for ourselves but also making sure customers that's applying those technologies, we have to keep them
safe. That's right as well, and keep their customers safe. We don't, you know, think about the early debacles people misusing chat GPT. There was a lawyer that had the tool right assummation that referenced cases that simply didn't exist exist yah, and he didn't check. And now, oddly enough the judge doesn't go for the oh the software did it answer? Yeap. You know there was another case in Canada actually with their Canada Canada one. Yeah,
you remember you know that one? Yeah, no, I've paid close attention to that. Where we'll tell this anyway. It was a chat bought that uh, powered by an LM that was answering questions about somebody when needed to make flight changes for a particular reason. I can't remember which reason it was. And they said, oh, yeah, there's no feast for that,
that's fine. Yeah, And so they were one hundred bucks you could yeah, yeah, anyway they went or he was supposed to get I think he was supposed to get a refund, and then when he went to get the refund, they're like, oh, you know, we don't rEFInd that. It's like, well you told me, yeah, like, oh, that's that's a mistake in the software. Yeah, And they actually went all the way to court on it, and the court ultimately said they have an
employee had said that you would be liable. You just because you use a tool to do something that employee might do, it doesn't mean you're not liable.
You've got to payback good on them. Yeah, it is good, and you know, it kind of highlights an important aspect with building these types of applications, which is, uh, you need to capture those interactions that happen between customers and users and this technology, these lms, just like you have and say any customer service scenarios that you always everybody's conversation be recorded. It's trading purpose exactly right, you know what you must do the same.
Even more important because you're having a computer talk to you're experimenting this new technology, right, Like I think we scrutinized customer service calls much more closely when they were new than we do today. That this is new, so it needs to be closely screwedin. You've got to make sure that your customer service agents are on script and giving the correct information the same as you do with
your lms. And this is a key kind of workload that we're finding with Cosmos these days and building these types of applications is Yeah, you can in the past have used these things to build chat applications and chatbots and also just chat assistance where you're talking and chatting with a human. Even more important now where you have lms that are generating responses sometimes had a whole cloth if they're not properly grounded to the data that they should be, you need to capture
those interactions. If for no other reason, then you need to be able to audit that information. But you can take that information and also feed that into maybe an mL model and do some kind of sentiment analysis. You can feed that into some kind of clustering. You can do all kinds of analytics on top of that after the fact, to make sure that your data is better, make sure that the information it's better, the prompts are better.
That's super important, and you can't do it unless you capture that data to begin with. Yeah, and so this has been a key workload for us for Cosmos over the last eighteen months, is using Cosmos as part of building
that chat interface and capturing that information. And now with build, we just announced our native vector search for Cosmos dB as well as something I'm super excited about, which is this new suite of technologies called disc an N. So an N is approximate nearest neighbor, right, and in the vector space. That's essentially how you do these types of queries is you don't do a keyword
match or a full text match or search. You use vectors and vectorize data and words and whatnot to to find the semantic meaning and al also find those related terms within there. And so now we've got this capability built natively into Cosmos. So customers that are using Cosmos to say, store and capture all that chat interactions can now do vector search vector database correct, so you automatically
have a large language model or a small language. Yeah. And now not just for the chat but also you know, vectorize your data like RAG pattern is a big thing now, right. You want to be able to have an LM that you don't need to necessarily retrain. You can provide it and augment that information that it generates for a user using data out of your database. And the best way to get relevant information is using these vector search technologies.
So take your data, your transactional or whatever data it is, vectorize its stored in cosmos, and then do a vector search over that and then have it enrich and augment. I like things like operations manuals, like documentation about the business, about how we operate. Those kinds of things are easy and obvious. But there are other things too that you might want to vectorize that you might not be thinking of. Yeah, like things like operation manuals
and static documents. I mean you can do those, you can put them in like a cognitive search or AI search. The harder challenge is how do you do that over high volume transactional data. How do I do that? If I'm trying to capture I don't know, actions in a shopping cart, or what pages people are browsing or other things where you have an enormous volume of data vector databases in the market today, and even search engines capable of
capturing that kind of data. You need an operational or a transactional data store, especially a scalable one to be able to capture that. This is what was unique about the announcement. And I'm not sure landed it build here, but the disc and en technology with Cosmos really allows you to take and build these types of scenarios over high volume transactional data and you just you can't do that with existing like ahnsw types of data. We used to think that the
OLAP cube was like the end all be all of finding related data. That wasn't you know, easily accessible by regular RDM myths. And now this takes too all new level, doesn't it. You need? So really all you have to do on that, I say all you have to do, but if you use Cosmos dB as you're a data store, now all that data becomes searchable, yes, and in fact natural language, and not just within
Cosmos. But also you know, we have a feature, you know, fabric is a brand new YE or analytics one like data Live platform o. The word fabric has been reused a couple of times, but I did notice the logo on my shirt to yeah, that's the new fabric, the new
fabric. Yeah, but we have a we have this feature called Cosmos Mirror which will take any data you write into Cosmos and then automatically etl that into your one leg So if you want to take that data, yes it's already vectorized, but you can store it in the one leg and then spin up a new workspace and create new mL workbooks and you can do all kinds of stuff like maybe do basic clustering in there, to do sentiment analysis over what
your customers are saying everywhere. You can go anywhere with this and do it in a big data analytics platform to augment all of that in there and get those insights and then use that to feedback in and that I think is the promise and what's coming next is you can build these applications. Customers can come and use them and they can you know, you can ground your data and make sure it's providing good, sensible answers. But you really need is that
three sixty. So I want to take that data, I want to do analytics on it, and then I want to apply that back to help refine the system prompts that are within there, help refine all of the all of the knobs and dials for how those things interact with customers in there, and help optimize those things. Boy, that's just creating a whole bunch of new jobs, isn't it. I mean, if you think about the expertise required to do that, you know and do and hit those tweak those knobs and
dials. You know, that's that's like a whole new thing. Yeah, but you know what it's Microsoft. What do we do is we take and democratize things that are hard and make them accessible for developers. Right. Yeah, you mentioned one like off the cuff there and we've been talking about a run ass because we talk more about data analytics over there. But just as
it's one drive for data. Yeah, right, Like you think about when you want to build a model like this, you need to pull from a bunch of different data sources and so you know, you need different considentials for every SQL server. Like just even finding all that stuff, it's a nuisance and one like really comes at it. Of all the data is here, and it's not a repository like you dump off a copy of the data here. It's the data so it's being continuously updated as well. And it just
makes a consistent way for resources to get access to data. So like the evolution of the data lake, isn't it. Yeah, And in fact it doesn't have to just sit there, Like you can analyze and run jobs on this data that's in there and then use something like a Spark to write it
back into like a cosmos or to a hot store or something. It should also be available in the lake, right right, it is, right, so it doesn't have to just sit there and just gather dust, right, you know, take and analyze and process that data and then feed it right back in again. Sure. Wow, yeah, mind is being blown well here this week. Just wait, it gets crazier than that. I did this show with a run already on run as and I'll put a link to
it. But you can put symbolic links into the lake to data stores that are not in Azure. You can link to Oracle databases. You can look up to S three, so all of the data appears visible in the lake, even if he's even in the lake. But it has the credentials, so if you have the rights to access that, then it deals with the credential calling to go over to a WS and pull the data as part of your search. Like oh so check out this big fish that I pulled out
of Lake Superior. Oh no that wasn't lake. You'reon whatever, like whatever lake you want. You're good, right, You're fine. Before we get to the break and we're it's coming up. But Cosmos dB when we talked about in near early days was really about h geodistributed high velocity database and that really to me said certain categories of projects are going to need this, Like, so is that still the case? Like does it make sense for most
folks? You don't want to experiment with copilots that sort of thing that they would go Cosmos, You know, it was never that was never the first foot forward, right. It was always where you could take it if you needed to write. That was always the promise of Cosmo says you could it could you could use it for any size workload. But if if you needed to do stuff that you could not do with a relational database, Cosmos could
take you there right right. You can always start small, and in fact, a lot of the engineering an effort we've been putting focused on is really designed to help improve that developer experience starting at a small scale. You know, we have a serverless option now and actually just recently we announced the availability for being able to take and seamlessly migrate from serverists to say an auto scale and then if you want, you can go back again in there. But
that makes it easy if you start with like a serverless option. That's kind of how you should do your dev test right, Like why pay for a database? Why pay for compute if you're not using it? Right, it's a dev right, or it's a test environment when you go into production. Great. Then migrated over to an auto scale allow it to elastically scale in
and out, and that's great. Other things as well have always been I guess, you know, for designing for a scale out database can be a little tricky, you know, yeah, absolutely, Then I was perfectly happy with that viewpoint that. Look, I've tried to scale document data stores across multiple sides. That's hard, and you don't want to own that. It's plumbing, right, So the idea that there was a knob in Azure around
causes Dbut that's put it in these geographies. Keep these syncs at this rate, like you pay for it, but you don't have to own that complexity. That's right. But it may begged the question like do I ever want this in one node mode? But you're describing one node scenarios, you're at least initially knowing you would go to multiple nodes they if the case arise, it always starts small, yeah, right, So, and it needs to be approachable and easy to use, and also it needs to be mutable in
a way, right. Like, one of the things that's been challenging or is challenging for building distributed scale A databases is you typically have to make very very complex design decisions very early on, like how do you partition data so that it scales out? And in the past that was always a challenge because in many cases you were asking a developer to predict how it was going to be used once it was in production. Only thing worse than that prediction was
trying to change it afterwards. And that was the thing that we realized is we needed to make it easy for a developer to change that because you guys have all done this for a long time. You all know you get an app into production and you're surprised by how customers use it, and then you need to go back and back to the drawing board and I need to refactor a bunch of this stuff. And that from a scale out point of view, if you do not have your partition design correct, you cannot scale.
It is the key to scaling, and that's another recent innovation is we've made it way easier for customers to go back and say, give me a big, great easy button that I want to push and allow me to repartition my data so that I can scale right and then maybe change it again. Like absolutely, the data size grows and new features come in, you need to repartition or have multiples. This is another innovation as well, is providing a
material easy way to do a materialized view on that data. This is common with like say a CQRS type of pattern right where you have ingesting of lots of data and you need to be able to scale that out. Well. Rarely the case that you're going to read the data the same way you write
it. You may have data coming in you want to do reports by state, or you want to do a report by some category, or you want to do some kind of dashboard over here, and you can't run queries over a large scale out database like that and get good performance on your repath. You need to have that data kind of repactored and pivoted so that you can
do efficient dashboards, efficient reports, whatnot. So providing that capability just another big read easy button for developers you don't and then so you you know, I do not like pre optimizing. I've been burned by it too many times. And I'm not saying I want to have the problem first, but I kind of want to have the problem first, right because often those optimizations create
more problems than they solve. So it's better to watch the data flow in and see where it's getting rough because you've got good instrumentation and go, okay, well if you're protuction like partition like this will reduce the impact, and then you try your benchmarkt like now you're describing as any area where I'm willing to express now to say, okay, let's try a partition strategy and see
what the instrumentation. But this is key for a developer experience, right is you need to have that flexibility to be able to come back and approach it in a different way or solve it in a different way. Folks ask us for a lot of certainty, and we don't have it. Man, Yeah, yeah, I appreciate you state. It's like I don't have to decide that right now. Let's wait till we see how the data flows before we decide make a decision, and then have the flexibility to revise. Yeah,
we guessed here, it's a guess. We're going to measure it like this and revisit this at some point and maybe adjust it. Yeah, yeah, well I appreciate it. So make it easy for developers to run a small scale they want to run on a single note. That's great, But help them make it easier for them to achieve that kind of scale that you just can't get that anywhere else. That's why customers use us. It's always been Microsoft's mo And with that, I'm going to interrupt for one brief moment for
this very important message. Attention dot net developers looking for the ultimate SDK to handle a lot electronic document processing. Meet tx text Control. Txtext controls your go to solution for seamless PDF generation, secure electronic signatures, and efficient digital forms processing, all within your dot net applications. Empower your products with robust document management capabilities, boost productivity, and deliver top notch solutions to your clients.
Trusted by developers worldwide, including me and Richard, txtext control is the SDK that makes a difference. Check out demos dot textcontrol dot com for live online demos and see it in action. Hey Carl here, tired of the debugging grind, Let's talk about a breakthrough that's here to help ray guns AI error resolution isn't just another tool. It's like having a seasoned developer right by
your side, suggesting fixes and explaining errors in real time. If you're curious as to how it can smooth out your debugging day, visit raygun dot com slash dot net rocks that's raygun r A y g u n dot com slash do O t n E t r O c k s for your free fourteen day trial to explore the way reygun is revolutionizing debugging. And we're back. It's not a Rocks. I'm Richard Canvill. Let's call Franklin Yo. Chat with our friend Mark Brown in the latter stages of the twenty twenty four build
in Seattle. Gray day as usual, Seattle. That's how this place works. It's I mean, it's cloudy and gray. It's just another day. It's just another day. We call that Thursday. Well, I mean it does seem like these LM workloads were made for you guys, Like, but what's the shape of the data that they're typically managing. It's the vector data stores, right, It could be anything, right because as a Jason data store, we largely don't care. Yeah, you can take anything, right,
You can have hierarchy, you can do whatever. Yeah, it's it's whatever. You need to go back to why we're in the space and what we were doing. You know, vector databases are super hot, and there are a lot of them out there now and they're getting a lot of attention, and they're very important to building efficient workloads on these alboraks got an implementation of a vector store inside of it now or will they will? Yeah,
So they've been doing a brute force thing lately. So you just basically put an array of vectors into a into a into a column and then call a cosign function on that thing til you can get basically it's it's an exact match, right, or a brute force not doing approximate nearest neighbor. So I got a story for it towards the beginning of the year, when you know, before any of this was available and nobody even knew how to make a
vector database or what they were. I was approached by a colleague to put together some training on AI and put you know, how to use chat, GPT, how to use the tools, the APIs and and and it was just because of was such a hot topic. And I said, after a while when things changed so rapidly, It's like, you know, this stuff is changing so fast, this is not the time. We just got to wait, got to wait this out and see what major tools shake out,
because they will be available. And turns out I was right, Yeah, yeah, it only gets easier. Yeah, all these things that we had to Can you imagine going into a training class like in I don't know, right before Windows came out, and somebody's teaching you how to write your own windowing system, right, and the Windows comes out, It's like, well, what the hell was all that then? You mean like prior to like Jeff Richter's book or something like that, Like, please give me some MFC.
Yeah, I'm old me too with you. It's been crazy for the last eighteen months. The everything, all the APIs for open Ai and everything built on top of it and around it, whether it's things like a semantic kernel or a lying chamber law index, these things are changing like weekly. They would go and push new builds and you'd break half of everything that's right
there. And a lot of customers I was working with were frustrated because there's a lot of center of gravity around using these orchestration SDKs for llms right, like, they do provide a good kind of single sheene if you will try
and help connect all these pieces together. But man, they were changing so fast, so fast that it was a lot of customers are like, you know what, I'm just going to set those aside for a bit because I'm trying to get something built here and if my billd breaks every single day because you guys are changing, then I need to kind of take my own ownership
of that. Ryan McKay and I were doing a YouTube show called The AI Bought Show, and it just kind of was like that, you know, he would and in fact, he would go and test some things on against I don't know if it was chat Gypt or just something in open AI.
Maybe it was like the you know, the playground, I think it's called right, and he would test it all and then get the response, and then we'd record the next day and the response was completely dead, and it's like, oh, they fixed this because he was showing like how to break it or whatever. Now it doesn't break. And the next day it was, oh, well this kind of didn't work yesterday. It's amazing. It's
just just goes to show you, you know, just calm down. This is a good time right now to investigate what Microsoft is doing because they they are the tool builders. They are the tool buildings. I'll pull some links for like lang chain and the other semantic kernel. This is something that we haven't really talked about all that much on dot net rocks. Maybe we could
just give us an elevator pitch for what that is. Yeah, So Samanic kernel is just among kind of these group of these orchestration SDKs for building l ms and kind of the equivalent of these things that people may have heard of would be lang Chain. It's really the big one I think out there. La Index is another one of these things. And you know what they do is Python, right, Yeah, they all support like Python and JavaScript for the most part, like lang chain for sure. Samanic kernel offers a dot
net sdk. They also have a Python option in there as well. They may have a JavaScript also that they're in there, but anyway, you know, these things are changing so fast, they might have something else in there as well. But the value of what these things do is they provide a way to easily kind of plug in lots of different things. It could be different lams. You can build kind of an agent's on top of these things.
They all have these notions of kind of planners or chainers within there, so you can really build kind of these complex LLM workloads and do it in a kind of an easier abstracted interface, right, so you don't need to get down into the weeds of native SDKs to connect to all of these different vector databases or llms or embedding models. You can just provide kind of that
single user interface for that. That's the promise of these things. The reality for the last year and a half is they keep changing so fast that it's hard to keep them and use them. But that's their promise, and I
think they do a good job of doing that. They are starting to solidify, if you will, some level is a semantic kernel after these things that were announced to build considered like a low level tool compared to like I don't know, the co pilot studio or any of these other high level tools for sure. I mean those are look at the copilots too. Is kind of a low code way of going and building these types of of applications, for
sure. If you're going to go and roll your own and you wanted to be able to bring together lots of different bits and pieces of these things. You would use something like a semantic kernel or maybe a combination of like native SDKs and semantic kernel to go and build these types of things. So that's their value and being able to do that provide that pluggable interface, if you will, for doing these things. I can feel people listening, going agents,
What what's that? Why do I need one of those or a series of these? Are these like little experts in a particular silo that maybe talk to each other or maybe have areas of ex ortise. Yeah, it's exactly it it So let's just I mean, I could kind of give you some a rough scenarios. If I'm in an e commerce type of application and I want to do i don't know, product search or something or asked about products, I may have an agent specifically tailored to doing product searches. Right.
I may have another agent tailored specifically to refunds or to some kind of customer profile or updates or anything, I mean, anything that you need. Like, all of these things kind of operate around the notion of you have a system prompt that kind of defines the behavior and also grounds it to specific data.
Right, And So the idea of the agent as you've created this kind of encapsulated whole that takes inputs and is specialized around a particular thing, maybe grounded to some specialized data, and then provide specialized outputs for the thing that Richard hates the most is that's an easy way to answer promorphize an expert software thing. You can give them their own personalities and you can tell them to answer every question like a sixties stoner or something. Yes, you know,
okay, dude, let me tell you what's going on. I did something very similar. I created a demo that I've used at the Fabric conference that was at actually it was a night last year, and it was built and grounded to the old old Adventure Works twenty seventeen data based from SQL service. So it's like a bike store, right. So I created this AI agent and I called him Wheelie, and I went to the Being image generator and
I created an image looked like it looked like Clippy. Ironically, could have like a little like a little cartoon avatar on a bike doing a big wheelie or whatever, and he had a big smile on and stuff like that. So I created a whole persona around this thing. The T shirt that said keep on truck, and I should keep the T shirt for this thing keep on wheeling or biking or whatever. But no, but it's a retail bike store, right, I want to create an AI assistant that's really into biking.
So the system prompt for this thing is you were super enthusiastic about biking, and you really just want to help people find all the biking and accessories that they need so they can go have great biking adventures. Right, and
you should have all this energy and all that stuff. So what's interesting is, you know, I go when you start testing this thing, and it's exactly what it comes back with this And I find if in the prompts you use word like brilliant you know, or just like master of you know, it will actually be smarter, right in terms of you know, or the
perception will be smarter. So you can provide all of that stuff, you can give it that personality, you can you know, it's a great way to represent your brand if a way, right, if you have kind of a specific vibe for who you are. As long as it doesn't tell you this refunds that don't exist. Yes, that's right. Accuracy is a feature too, and safety. Here's a coupon. Yeah, it's it's interesting to you know, we did I did this great show with Steph A. Warre
as we talked about it. It's a language computer. It fits out language for you based on your instructions. Yep. Ye, And it's just humans that project understanding into that. Yes, you always have to remember your instructions and also the data you give it as well. Right, it's only smart enough for whither what's been trained for or the information that is provided. Well.
Yeah, and it needs testing. You know. The conversation over and over again is that you need experts in the data sets to test it and say, oh that is correct, that's incorrect, and sort of work your way through it. It's it's it's certainly not a reason not to use it, but it's like it's just like if you're gonna work in a dynamic language, you need testing around it. Right, you know, you you you have to do the whole thing, You have to own the whole problem,
and you can make these things work and be incredible labor saving devices. You know, they talk about response times dropping down to seconds and you know, making customers happy as long as it's correct. I had this argument with a friend, you know, when I was talking about hallucinations and stuff and wrong answers, and the argument was to defend this behavior. Well, I know people like that too. It's like, yeah, my uncle Joe is a complete bs er, but I'm not going to hire him to run my company.
Joe's not processing my refund. No, those aren't the folks you use. So now fair enough, Like, what do you want from your software? What are you expecting? I just wonder for you know, trying to imagine the next couple of years in the transformation of the user interface, like a lot less buttons and knobs and things and a lot more just to free flowing texts. I think it's the natural language processing is kind of the going to be the interface for these things, right, not typing your talking and
learning. Yeah, I also wonder about machine learning models for vision, Like I've always thought it would be great to have an assessment of frustration on the operator's part, right, but the software could see that you're struggling with it, just you mean, like feed in a like a jiff of that guy banging the hell out of keyboard or something anymore like most of these machines have a camera on already. Yeah, Like, could I actually recognize your struggling
to achieve something in the software and then affect the software? Like do you need help? Can I put up a prompt? You know? Or I've noticed you only use these six features? Could I put them front the center? Like? What if the UX was dynamic to the way the user used it? Or please place that hamburger in the bag before I call and keep this light for somebody to come over. Oh yeah, So I heard someone on in one of the keynotes, and I can't remember who it was say
that many people would rather talk to an AI than a real person. That's the first time I had ever heard that argument, because in my part of the world, most people don't want to go to a website and chat with a bot, you know, they want a real person. But is there this growing population of people who would rather speak to an AI than a real person. I guess it depends on the capabilities of that AAI. Let's start with the initial thing is would you rather go to a website to get the
information? Would rather call the company? Right? Like I tend to not want to pick up the phone anymore. Because it takes longer looking for the least amount of time to get the things I need. And so you know, the reason you've been avoiding calling tech support and things like that is it takes a while and you have to keep explaining yourself in some ways. If the software is the interface for that, it's less frustrating and you're talking usually
to someone who really doesn't understand what you're doing. But it's just taking your answers and plugging them into an expert system and anyway, you know. Anyway, so what you have as a transcribed version of a chatbod, like, why is that absolutely right about that? Yeah? I think it's gonna whether they want to do it or not is going to depend on how good the experience is. Okay, well, here's a great example. Things that I used to Google for I now ask chat gipt. So you basically believe in
the I Feel Lucky button on Google because you're just getting on answer. Yeah, you're talking about school search. It's you know, you put in a search term, you look at the array of answers, right, You're like, I'll let me rEFInd their search term, right, but I will ask a question like, you know, is there a coin operated laundry facility at this hotel that I'm going to. It's much faster and easier for chatting through a site for that, the calmbing through a site and trying to find it,
you know, or calling them or definitely calling them. Yeah, I use the chatchipt on being like maybe five six times a day. Yeah, it's replaced my search. Yeah, it's also replacing things like Google Translate, for example, because now with four to oh and not for zero, but four oh chpt four oh. You know, you can use your voice and you can just talk to somebody and it will say, all right, translate the English to Spanish, the Spanish English. We'll just do that. Yep.
It's powerful. It's powerful. Stuff's actually I always had a conversation with someone's adjacent but not the same thing. Whose husband is waiting for a cock clear implant. His hearing has gone that bad. He says, I don't even want to talk to him or I have to shout and I always sound angry. And I pulled out Live Trends, I grabbed her phone, installed Live Transcribe on it, and then we just kept talking and she's like the words are coming out. It's like just put this in front of him,
like, what are you doing? You've had the tool in your pocket the whole time and you didn't know. Isn't that funny? Yeah, But the fact the idea of walking into a shop in France when your frendsh is bad, my friend is bad, and being able to just have it spew frenchise Frenches just slightly off. Yeah. They maybe bring the world closer together. Yeah. Yeah. It was a while ago that Google did some keynote thing where they were where they were doing live transcription and translation and voice and and
you supposed to get a product for that, but it's really good. There's a browser plug in for Google. Well that's the truth. I think it's now no longer you're not going to buy dedicated hard for that. Your phone will do it. Yeah. It just is a thing now, it's a thing, a new capability being added to the phone again because it didn't stock like, it didn't do enough. So the doomsayers will say, you know, all these things are being replaced by a single app and a single interaction.
What you know is everybody else doomed? You know, is this so disruptive that it's going to cause mayhem? I don't think it's anywhere disrupted than anything else. Yeah, I agree, you're appreciating the choir, But you know, there's certainly people out there who were You know, the story of the lad Heites, you only go as far as the Lades were actually a group of people who very technical people by the way making They were hand manufacturing
cloth, right with very simple tools. And in came the first steam powered looms and they smashed them up. And the thing they never talked about after that is after that the employer came back and they talked through how they were going to train, and all those folks started running those automated looms. And the biggest thing to happen is, yeah, you need a fewer people making cloth, But cloth became so much less expensive that people bought more clothes,
right, yeah, and then many more people employed. Well, the term ludite is thrown around as someone who doesn't understand or like technology, and that's not the case. The Luddites were very technical. They were they were the high tech, but they were they were frightened by the change, by the
change. I have a more recent example, and that is just the kind of the emergence of cloud I used to see a similar kind of reaction from your old I T pro guys that were out there in the loading dot grabbing all those racks of hardware and then going and racking and loading all that stuff in, and then they're like, wait a minute, now that's my job. My job is doing that. Now you got some guy with a web page pushing a button. You have friends that are fit that I think you
know, I'm talking. We have several front and I told him, I'm like, look, I go that piece goes away. But there's a whole ton of other stuff now that you were going to learn and do, and you were getting to the bottom of your to do list, right, That's true. It's not a thing. It's not a thing. There's so many more problems to solve now. I got to I say this on every dot and I rocks show we've done since we've been here. And I'll say it
again. The chances of you being replaced you as a developer being replaced by an AI are slim to none. But the chances of you being replaced by a developer that uses it are high. Not high maybe, but significant enough so. And there's no excuse for developers there's no you know, there are no safety concerns about using an AI to help you write code, to look things up, to do research. You know, they just don't ask it.
Whether you should ask your boss for a raise, you know, that's that's not ken If you want it's just the you still have to live with the consequences, that's right. Yeah, you may get bad advice, but you won't get bad code. And if you get bad code, just ask again. Yeah, try it again. I've been here's a trick, ask it to comment your code that it generates. I'm all in. Yeah, the last eighteen months I even used it, and the good HELB copilot is
a great example. I needed to I needed to retrain learning Python. Yeah. I've been a c sharp developer for you know, as long as it's been around in two thousand and two, I guess right, and even before then, of course other Microsoft technologies, and I go back to VB back in the nineties, but me too. This AI space has put a lot of focus on Python right because, and I see this with the customers.
A lot of these projects will start within kind of the data engineering and machine learning space because it seems to be that's the center of gravity for a lot of this machine learning and this space. It is a great language, but I didn't I was not very fluent in it, but I needed to learn it. And I've been using gethub, copilot and the tools in visual Studio and vis code and elsewhere to help me retrain, to help me retool myself
so that I could be more productive writing Python. So you have this assistant effectively, that's coaching you on writing your running better. Take this C sharp code converted Python. Yeah, tell me comment it, tell me what it's it's actually briant. You know what I do now is I write a comment as to what I wanted to do, and then the Python comes out, and then it shoots out Python. I'm like, yeah, that's great. Yeah. Yeah. I can write the code and have it commented, or
I can write the comment. It'll write the code. Yeah, you have a choice. Is that like schema first versus code first? Yeah. I write the comment and it writes code that also has comments. Yeah, ma'am, I used it, and I told you this. I used it for you know. It would spit out link statements that were some not readable by my customer, and so I basically asked it to you know, split this out into multiple for loops or whatever and comment it. And I took that,
commented it. Yeah, and so you know, above the liuery, above the link query, this is what this link query does. Great, So not only is it solving a problem, but it's educating my customers, right. Better documenting, yeah, much better documentation. Yeah. I think it was twenty twenty two. My GitHub annual reports that the primary language I was writing in that year was Python. Really, I didn't think I was
writing that much Python. As you know, I munch a lot of data in my life and buy golly pythons a bunch of data and so just you know, use the tool of them. I mean, the second was YAML, which makes me sadder, and the third was c sharp. Thank goodness. Yeah, I'm sorry for your YAML. Yeah. You know, I run a lot of infrastructure too, So Yamil's my friends there, you go, that's a good job. Yeah, oh well I got I got that
they complain about. It's all good, right. And then it's also so this reality, it's like you are not the language you programming, right, You're the person who solves problems, and you know, the more tools y am your toolbox the easier gets that is the key is a ye. Yeah, yeah, I'm It's been a long time since I cared what language I was working. I'm just trying to get to the bottom of the problem. And now the tooling is super good at helping me. You know, why
am I struggling with this? You know? And and it'll it'll get you over the hurdles. This is why developers, if for no other reason, should embrace this. Yeah, right, because it's a great robber duck. Holy man, it's such a good rubberd to make so much at very least, yeah, so much more productive. Yeah, I spent much time testing, less time just debugging stuff. I'm getting it just front to end.
It's getting done faster. And here's the thing, like, you know, I don't be doomsday, but you know, if you don't embrace it, you may it may be to your detriment. So why not we work in technology? Why should we not be embracing that? Sure? Why are you using an it like you could just be? When you got into this gig, you knew it was going to be changing fast, right, that's what got you excited about it? Well, yeah, here it is again.
We're turning the crank and it's changing fast. Using your chromium tip tweezers to light up electrons anymore either, right, like listen, shout out to right. Not organizing electrons one at a time anymore. I can hold B zero THEREO one zero one zero zero, just another tool, more automation, more productivity. Right, and you do. Maybe we'll get a little further down that list. The joke of courses. Every time we get these things, list gets longer. Yeah, there's more to do. There's so much more
to do. Mark Brown, thank you. It's been a great pleasure talking to you, and this is great stuff. I love talking to you guys. Thanks for having me back on. I'm glad I could be your closer for this build for it absolutely. Thank you friend, all right, and we'll talk to you next time on dot net rocks. Dot net rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut,
and of course in the cloud online at PW O P dot com. Visit our website at d O T N E t R O c K S dot com for r s S feeds, downloads, mobile app comments, and access to the full archives going back to show number one, recorded in September two thousand and two. And make sure you check out our sponsors. They keep us in business. Now go write some code. See you next time. You got jad middle Vans the