How'd you like to listen to dot NetRocks with no ads? Easy? Become a patron For just five dollars a month you get access to a private RSS feed where all the shows have no ads. Twenty dollars a month will get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, Carl and Richard here with your twenty twenty four NDC schedule.
Will be at as many NDC conferences as possible this year, and you should consider attending no matter what. The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Tickets at Cphdevfest dot com.
NDC Porto is happening October fourteenth through the eighteenth. The early bird discount ends June fourteenth. Tickets at Ndcporto dot com.
And we'll see you there, we hope.
Hey, guess what it's dot net rocks again. I'm Carl Franklin and I'm Richard Campbell, and this is episode nineteen nineteen. And Buddy, I went and I looked up you know what happened in this year in history? So some significant events included the collapse of the Habsburg and German empires.
Yes, the end of World War One, end of World War.
One, the redrawing of the map of Europe at the Paris Peace Conference, the influence of Woodrow Wilson's fourteen points.
Interesting they call it the Paris Peace Conference. It was the Versiah Cords, and they were not They were not very.
Peaceful, not very peaceful.
Yeah, they were a very good setup for World War two, yep.
And sowing the seeds of World War two, growing demands for self determination by smaller nations, the growing involvement of the United States and European politics and trade all that. Yeah.
Yeah, one would argue this is the ess of the nation stage. Yeah, basically the model we've been functioned ever since.
I bet you didn't know Richard Campbell knew so much about history. O. Who am I kidding? You knew you knew? Go ahead and roll the music, because I got a pretty good better no framework, awesome? All right? What do you got? So you know, I'm cutting down on travel to conferences. At the end of the year, I'm not going outside the country. I'm not even sure that there are any more domestic conferences that I'm going to at the end of this year, but I still want to do some Blazer hands on training.
Right.
So what I've done is if you go to APPVNX dot com, slash training or nineteen nineteen dot po dot me, you'll see the training page which links to a form that you can fill out. I've decided I want to do these online classes on Mondays. Oh yeah, mostly Mondays from now until the end of the year, most of them except for you know, some week off here and there. And this is a form where you can select the
dates that you want to attend. Each Monday, we'll have one starting at nine am Eastern and nine am Pacific, so that you know, if more people want to do in a Pacific time blah blah blah. But the deal is that once we hit that magic number of ten, then the class is going to happen, Right, So you should select all of the dates that you can possibly make, and then there's a greater chance that one of them.
Will will take But online classes.
Will online classes, but you know it's interactive. You'll be able to do the hands on work and even though we won't get to all the materials in one day, you still get all the materials right so you can go back on your own time.
Well in the class size sounds like assuming nice as small too, so people go be able to interact a lot.
Well, ten is the minimum. I haven't really set a maximum yet, but of course if worry about that problem when it happened, I'll worry about that problem. So that's what it is at Phoenix dot com slash training. Who's talking to us Richard crawbd to.
Comment on top of show nineteen sixteen, So just last one published. To be honest, this is the show we did with Mark Rendalls. We called how Simple is as Simple as Possible? And in finding conversation with Mark, as always he's a thoughtful guy as well as very funny. I think we spent most of the show laughing. And Rob King has this great comment. He said, once again another great episode which reaffirms my position that everything old
is new again. I feel like there's a whole generation of developers coming up with only known JavaScript frontends and think the server is only for APIs or serving the JavaScript. I've started going back to a server side first approach of building apps, keeping get all in the server and then build it out towards the client only required. Yes, Lazer made this is really simple, and it's approached to
adding interactivity as required. I've even converted a Blazer WASM app to pure service side because it's actually faster than the first page and the app doesn't need any interactivity beyond the forms. To that end, I've found that you can integrate htmx something that Mark talked about right seamless see into Blazer server and a built out of a little demo repo which I'll include in the show notes
to Rob's GitHub repository. It's just a sort of hey, these two things will play together, Blazer and HGMX.
Yeah, there you go.
You know, it's just sort of a sort of recognition of yeah, we're kind of calling in a cycle. Like there's clearly cases for putting more load on the client, but one would argue we swung too far that way. Yes, although I know as we get into these conversations about large language models and machine learning so forth, there's now a big push about how do you get more of that workload on the client because folks are concerned now that too much of that workload is on the server.
So I think it'll be some good talking points for today's show.
I'm really happy that a guy like Mark Render, who you know, go back and listen to his stuff before a Blazer came out. He was not really a service side guy at all, like very much in the in the composable c I CD pipeline kind of thing and and all you know JavaScript, and a guy like Mark comes out and says, you know what, I took Blazer server for a ride and I really like it.
Yeah, it's really great. Well, you know, and again his whole, that whole how simple is simple is getting back to the intention of what we're trying to do for the customer exactly right, Like I do think we get enamored of the technology. And again I brought this show and these comments out because we're about to go down a path about a whole other class of development, and I think it's good to push back on those fundamentals and say, what are we trying to deliver to customers?
Yep, no pressure pre shant, Yeah for sure.
So Rob, thank you so much for your comment and a copy of Music co Buy. It's on its way to you. And if you'd like a copy of Music co By, I write a comment on the website at dot at rocks dot com or on the facebooks. We publish every show there, and if you comment there and never read on the show, we'll send you a copy of Music to.
Go By, Music to Code by, Still Going Strong. I just sold another Flak collection, which is good, and I got inspired last night to do a new one.
How many how many have you made?
Is it like twenty one?
Twenty one? Okay, so you know you're digging around for another track? Huh, that's awesome.
Well, I was listening to it last night. I just decided to.
I don't know if you've heard it. Stuff.
It's pretty good. It is pretty good. Yeah, I've been listening for a long time. Yeah, I mean I started. I was listening to the more recent ones okay last night and I was like, yeah, you know, this is pretty good stuff. I think I got inspired to do another one.
Hey, you should meet that guy. He's pretty clever.
Yeah, so we'll see. I'm not announcing it now, I'm just announcing my intent.
You got it?
Some itch?
I like when you have an itch? Yeah, stuff happens when you have an itch. Yeah yeah.
Well, anyway, you can also follow us on ex Twitter. We've been there for years, but the cool kids are hanging out. I'm Astdon, I'm Carl Franklin at tech Hub dot.
Social, and I'm Rich Campbell at master do Social.
Send us a two. That's another way you can get a copy of music to code buy. Okay, let us introduce Prashant Boyar. He is six times AI MVP, three times Business Applications MVP, Microsoft Certified Trainer, speaker, author, and leading AI architect from the Washington, DC area of the United States. Currently, he works as an AI architect in the Office of the Chief Technology Officer at Applied Information
Sciences That's AI at dot com. Braschant is on the leadership committee for AI Fest, AI and mL User Group and the Northern Virginia Data Platform User Group. As a renowned international speaker, he frequently presents a tech conferences. Additionally, Prashant was awarded the Antarctic Service Medal of the United States of America for his outstanding service in Antarctica. You know, I was going to tease you for not having enough stuff on your CV, but let's go right to Antarctica.
What was that?
So, first of all, thank you very much for having me. That was once in a lifetime experience for me. Like I grew up in India and two thousand and seven is the year I came to the United States for my master's degree and I got I was really fortunate that I got a scholarship in the university and as part of that scholarship, I got a chance to work with a lot of brilliant scientists. And those scientists were studying the costmes raise and everyone in that room except
me was like really brilliant. So I had to, you know, fit in and I know, and you.
Might be a little unbiased towards yourself.
Now, Like if you if you talk to any experimental physics physicists or in theoretical physicists, not only they know software development really well, but they also know all this machine learning match and some advanced mathematics really well. So I have like after that, I have really high respect for all these physicists because for the work they do. And if you compare the salaries against that, you will be surprised.
Yeah, they don't get paid much for yes, as hard as they as they work. Antarctica is a great location for doing stuff like measuring comic cause of rays. You know, the atmosphere is a little thinner, and it's very very dark, and there's a minimum amount of noise, but it's and it's sad. That's like the best time to do that work is midwinter of Antarctica where there's no sunlight for months, so for the tough place to be for shann you hung out there.
Yes, So as part of that experiment, like we built, it was a joint collaboration WITHWEN multiple universities, and ASA was the main sponsor because typically NASA sponsors this, you know, give this huge grant to the unisity so that they can do research on their behalf. And as part of the research, you know, the best results to get is via satellite, but you know, sending any payload to satellite
is very expensive. So they came up with an approach about how we send this really large duration balloons in Antarctic continent which fly around you know the south you know, South Pole for around twenty to eighty days, so not really twenty to thirty days, collect as much data as possible, and then we take years and years to analyze the data.
And not many people know, but US government has a really big station there, like Unartic continent is huge and a lot of countries have stations, but US in particular has like huge stations over there. And the station I went to, it's called Macmurdo Station. So when I was there into like in the end of twenty eighteen, sorry, two thousand and eight, I was there for almost a
couple of months. And that is like a typical summer time in Antarctica, and that's the time where they will have like around thousand people on the base and everybody will be there to conduct some kind of scientific experiment and to support those experiment. They also need a lot of people, like they need cooks, they need chefs, you know, they knew it, people who can like blow the roads and all this kind of stuff. And I met a lot of interesting people there who like, let's say a
software engineering by trade. They work here nine months in US continent, and for three months they just go to Antarctica and work as a chef or as a cook because they just like it there and the pay is good.
And the base has good internet access. I hear yes, and I know that because I have a friend I went to grammar school with who is messaging me from Antarctica. I'm like, what you're aware, but he was at that base and yeah, so pretty cool. How in the summertime in Antaro would you think of it as bomby? Would you go out in shorts? What's the temperature?
So temperature is not that bad. You can go out and go and go out in shots only for ten to fifteen minutes, that's it.
Wow.
So like the during the summer time, you know there the temperature was in the range of you know, like minus let's say ten degrees celsius to plus five degrees celsius. Okay, but there used to be some drays where you know, it was like very very bad, Like we had like one day where it was minus twenty degrees celsius and
things were pretty rough over there. But you have heated rooms and you know, like food is awesome, there are some recreational activities and you know, and that when you go to those kinds of places, you kind of realize, okay, how fast paced your life is back in like you know, you know, the civilized world, and how much of a time you got when you go to these kind of places.
Did you ever have to slice open a banta with your lightsaber to keep warm? No? No, that's the question.
I was.
I was fortunate enough. No, I went there on a good good time over.
There, Yeah, very good. You only had to smell them on the outside.
Oh my, sorry for that little diversion, but I just had to know.
It's great.
So let's talk about Microsoft co Pilot Studio. I know because I went to build with Richard, and we know that it's for building your own custom agents and you can connect them to a bunch of sources of data, and we know that you can hook them up into M three sixty five and all that. But what don't we know?
Okay, so let me go back to the history and start with Microsoft first, foray into the cornosential AI. Like if you remember in March twenty sixteen, I think it's the first time when Satya Nadela went onto the stage and he announced about Microsoft bought framework. Because still that time, there was no good story when it comes to building let's say, any kind of child bought or conversational experiences.
And after maybe two or three, after a month after that, I know Microsoft is getting a lot of good price right now because of their partnership with open AI. But they also had their fair share of failures as well. I think In March twenty sixteen, they launched a Twitter bought called day oh yeah yes, yes yeah, and its long form was thinking about you And they launched on Twitter and that pot was from Microsoft Research and it was their first public foray into the world of AI.
And people soon figure out on Twitter, you know, to teach that bot to say some nasty stuff and Microsoft yes, yes, yeah, Microsoft. Microsoft try to like you know again reteaching like good stuff, but within sixteen hours they had to take it down and they still now a lot of key executives at Microsoft talks about that particular failure saying like, hey, we learn a lot from that failure, like how what other
things not to do with AI stuff? And so they continue to work on board framework for next three years. But board framework, like is very complex.
Yeah, we did a show, We did shows on bot framework, like you said, like twenty seventeen. It's hard to think about that now because the opening eye things overwhelmed everything. Yes, and they made a lot of sense, like the fact bot to me was totally logical that you could point a piece of software at a fact and it would give you sort of a natural language interface to it.
But still, like Microsoft struggle to get really good traction with a lot of customers and developers because it was extremely hard to get you know, your conversational experiences right. And then in November twenty nineteen they launch a low code version of it called Power Virtual Asian and the name Power itself can give you indication which product family
belongs to. It was from a power platform. So they continued on that journey and then basically there was a time where the entire investment on board framework stopped and all the things like Microsoft was doing on Power virtualation side. They had decent success. But then the real success or real i'll call life changing moment for that particular product
or group was Ignite last year. Fen went on stage and say, hey, Microsoft now is the co pilot company, and if you would like to create a low code, no code co pilots, then Copilot Studio is the product. And I know Microsoft get a lot of flak, especially their marketing department, for naming like weird products, but I think this time really nailed it.
To be clear, GitHub came up with the name, right, That's why it's such a good name, because Microsoft didn't think of it.
We have a history of making really long names that are very kind of academic sounding.
Well they're named by Kimmittee.
Yes, yeah, and someone I heard. I don't know how true it is, but this product was supposed to have another name. They already have a lot of printed name and banners on it, but just three one week or two weeks before Ignite, someone came up with this name and they have to, you know, do a lot of you know, last minute stuff.
There's been a lot of name twitching, right copilot for Microsoft three sixty five, Microsoft three sixty five copilot like it's.
It's you know, the AI thing has been sort of like a gold rush, hasn't it. Yes, like every company wants to get their put their stake in the ground and say we're doing AI or we're doing you know, generative AI. And I mean I don't I don't want our listeners who are dot developers to be thinking that, oh, development is gone. Microsoft is just focusing completely from now on on AI. Because if you went to build, there wasn't a whole lot about dot net, you know what
I mean. There wasn't a whole lot of code, plumbing code. It was all focused on AI. But you know, we're, as Richard said in the beginning when he read that comment, we need to stay focused on delivering value to our customers. And the sort of AI thing just smacked this in the face and came out of nowhere and says, hey, guess what this is going to allow you developers to provide better value to your customers. So really interested to hear some scenarios about that, and you.
Write a call like, you know, just don't think that copilot Studio is the only product that I will need to use, you know, when I had to create custom copilots. There is another story in Azure world called AI Studio. So if you are a professional developer and you don't want to use let these drag and drop kind of tools or low code no code tools and you would have full control. Studio is your choice where since it's part of Azure, you know it has a really good
pro developer story. You can be a dot net developer, you can be a Python developer, and you can use all the good services in Azure. And one other thing I like about AI service a Azure EI studio is
the model selection. Like. One thing Microsoft has done which not many people know is Microsoft launched something called Model as a service where you can bring in some open source model as well and host those in Azure and use those in your Genia applications, so you don't have to go with let's say models from open Ai or models from let's say Microsoft for that matter. You can use Lama hugging pace. You know, those models are also available.
So if you want to create a co pilot or agent using that, you can do that now in Asure.
But they still have GPT four GPT four oh dally all that stuff from AI Open ai. But all right, so wow, So that is really the fundamental difference between these two is that co pilot studio is maybe for draggy dropping business people and Azure AI studios more for us.
There is one fundamental difference which not many people tay pay attention. So when it comes to building a co pilot or any kind of a application, a lot of time people don't think about the UX part of it. I know. That's where a copilot studio shines really well. Like out of the box, it has really good capacity of you know, having a really good UX. Let's say I launch a co pilot for my existing enterprise app. But then that copilot should able to understand or behave
like actual human right. The reason chad jipt got so popular so fast is because we can interact with it and you know, it can understand like at least what I'm trying to say maybe ninety to ninety percent of time, and it can give me good information back. And if you compare that with old version of the chatboards that
more of these public facing website used to have. Those used to be like, oh the major retailers or bank used to have those, like used to be simple time st where the first thing I used to google that time is ourbing is what's the best way to get to an actual human agent rather than you know, going through their automated.
Response because that's what everybody does.
Yes, But now with this, after the generative AI and some of the advancement, you know, you can at least have some decent conversation. And that's where this co Pilot Studios shine where it has a lot of widgets, it has a lot of you know, building capabilities that if I would like to build a good conversational experience, Copilot Studio really shines as compared to Azure Eye Studio.
Can I take the agents that I make in copiled studio and move them over into Azure Ai studio and enhance them.
Right now, the create direct conversation story is not there. But what you can do is, let's say, if you have already an agent in Azure AI side, you can directly call that from a Copilot studio, so you can do integration. But there is no like Magic vand to convert those.
That's actually better.
Yes, well, it also reminds you that there's more than one team instead of Microsoft working on this stuff, right, you know, that's that's sort of the real reality is that there's lots of different places moving at once.
And I'm a big believer of Fusion Team where hey, developers, you focus on you know, doing really crazy stuff behind the scene, you know, make the magic, but also gave a lot of information worker or business users and ability to you know, take the advantage of what they know from the business and then you know, have an easy way to interact with the back end system. So so Copilo Studio also have that functionality way. Let's say I have my logic app I would like to call or
I would like to call my EPI. I can do that directly. There is an STTP connector available or I can also invoke a power Automate cloud flow, and power Automate has more than twelve hundred out of the box connectors where you know, or I can build my custom connector as well, which then can you know, call my back end APIs and then can get the job done.
I have a quick story here. I got an email last week from somebody who said, you know, I see that you have a YouTube channel and you've got some great content there, and uh, you know, for a very low fee. Actually it was pretty good. I will, I will, I will generate SEO happy titles, descriptions, tags and stuff for your YouTube content. So I said, okay, sure, I'll try it. And what I got back was a RAR file full of text files for and a lot of
them were for the dot net rocks videos. And they kept mentioning the dot net Rocks videos are simply a still shot with our podcast that plays on it. And this thing went on and on about how you will learn from these training videos, hands on experience and stuff.
They got to the they got to the songs that my band has put up there, and they called, you know, Nachelle, who's our background singer, but she does sing on some of them, but on this one that I was singing like all of it lead singer Nachelle Rollins blah blah blah, this big, grandiose stuff, and I wrote it back. I'm not gonna say it is by her own back and say, dude, this is bullsh this is some AI generated crap that doesn't know a still video from you know, from a
training video for crying out loud. So yeah, I won't be using their actually incorrect, factually incorrect and being sold as you know, the I don't know whatever.
So I mean, you say, quietly said Fusion development there, which is that power platform working with dot net dev and so forth. So you see a sort of same dynamic where we have domain experts, which I think is the big biggest challenge when you're building a lot of these language models, is you need a domain expert checking for factual accuracy, knows enough to know that is right
or that is wrong. Yes, who actually sang on this track, as well as some coding knowledge to be able to pull the pieces together.
Yes, that is, that is correct. And one of the things is, you know a lot of people like really got excited after the rise of generative EI, but there are like legitimate you know application, A lot of organizations
built which doesn't use generative AI at all. Like one of my customers, they did use Copilot Studio, and they did launch a custom Copilot for their public facing website, but they didn't use any of the generative AI features at all, Like, because there will be some instances where you don't want to give a sort of correct or incorrect answer, right, You want to get give a factually
correct answer. And that are some of the things that are there in Copilot Studio where for some of the information you absolutely want to give the appropriate answers maybe from your own knowledge pace or maybe from your own database or you and that you can do. And there are certain things where you think, okay, I can use generative AI for some of the answers. You can do that segregation wasy very easily as well.
I'm just another story. I love telling these stories my Facebook feed. I don't know about you guys, if you're on Facebook at all, but every once in a while I scroll and I'm seeing these accounts that are something like tiny houses, right, and then this clearly generated beautiful scene of this wonderful house with multiple staircases and flowing waterfalls by a brook. And it's just like a place that you'd want to be, right And the caption is like peaceful place. It's not. Hey, this is where I
am right now, it's not. And to get millions of views and hundreds of thousands of comments, it just makes me mad. Welcome to the new world.
Well, and again it's you used a bought to generate the content. Who's to say you didn't use a bought to generate the hits? Well, there you go, all right, So let's take a break, and when we come back, I've got a couple of questions.
Great, did you know there's a dot net on aws community? Follow the social media blogs, YouTube influencers, and open source projects and add your own voice. Get plugged into the dot net on aws community at aws dot Amazon dot com, slash dot net. Hey Carl, here are you maximizing your dot net applications potential? Rayguns tools not only catch errors, they provide actionable insights to optimize your code experience the power of detailed reports and real time monitoring to enhance
your applications performance. Join thousands of dot next developers who trust Raygun to help them keep their applications running smoothly. Visit raygun dot com, slash dot net rocks. That's raygun, r A y g U N dot com, slash do tn E t r O c ks for your free fourteen day trial.
We're back. It's dot at rocks. I'mrogard Campbell. Let's call Franklin. Hey, Hey, I'm talking to our friend Forshett about the role of as your AI studio as well as the co pilot studio, because what's better than two?
And I like how you can use them together.
What's the call out for Patreon there, Carl.
Yeah, And if you don't want to hear those ads during the show, you can subscribe to our Patreon account for five bucks a month. You can get a feed that has no ads. So there you go.
Awesome, Forsham, you mentioned like the ux of a co pilot app. Isn't it just a textbox?
Not necessarily? Sometimes it can be text paused, but some times you may want to show a really nice adapt card. Sometimes you may have to show an image or maybe the user is looking for a video. So those kind of things like if I go with traditional board framework way, those kind of things are really hard to you know, get it right. And another thing way where Copilot Studio really shines is you know, like how I can support
my copilot across multiple devices or multiple experience. That is like one of the biggest challenges I have seen with multiple conversational experiences. Like, yeah, like I build it, it's worse on my machine, or it works in the environment where I have deployed, But what about now the customer wants to use the same copilot on Microsoft Teams, or they want to use it on a public facing website, or the customer loves Slack they want to use it there.
Like how I can you know, now redo everything so that you know, my custom copilot will work on that experience as well or on that service as well. So copilot Studio has a lot of these channels or services already prebuilt where I don't have to redesign my experiences. All I do is do some back end like changes in the studio itself, and the same copilot then can be accessible on let's say SLAG or Microsoft Teams.
In both of these things, I noticed that you know, you can connect it to your database. Yes, And apparently the coolest thing is you know how many reverse threaded left handed hemptas are left in inventory in Atlanta, Georgia warehouse, right, and it'll just tell you that kind of stuff. But the cooler things about generative AI for me are building
models from you know, training and unreal data. And is that something that is not necessarily as valuable to regular, run of the mill businesses as it would be to say, you know, a chat gpt, which is the genie in the bottle that knows everything.
So, if I understand your question correctly, you're alluding more to you know. How about I have my own data and I would like to use that data into generative AI. That's one way of doing it. Or maybe I have my own data and I would like to generate my own model using that data because I want the answers in a specific format or in a specific way rather than what you know, Chat Jupiter gives me. So there are a couple of things you can do. And you
may have heard this a lot. Very few companies these days are investing building their own model because it's very hard to compete with companies like Amazon, Google, or open AI or even Microsoft. So most common techniques which most of the enterprises are using these days is either you go with a rack pattern like retrieval augumented generation, where you bring your own data and then feed that context or data to your large language model and then get
the inside. Second very common is not very common, Less common and more expensive out is fine tuning this models like where as part of fine tuning, then I provide series of prompts some training data. So then after a fine tuned model is there, then it will be responding in a specific way, and it will be also having a specific information that let's say the generic large language model or foundation on a large language model will not have.
Now, when you say that, do you mean cleaning the data, like taking out the things that aren't relevant, or do you mean when you say fine tuning, do you mean telling it what to ignore in the data.
Yes, so you have to basically provide custom instructions. You have to provide custom prompts, and you can also provide your own data as well. Where okay, these are the sete of things that you should be doing in this particular way. Don't include that or don't include these, so those kinds of additional things you can do. One thing I do want to highlight is fine tuning is not for everyone it's only you know, for companies where you know you're going to have large workload or your consumption
is going to be very high. And let's say with this large language model and or this large language model will not work in its current capacity because under the hood, if you read the licensing documentation from both open ai or even from Microsoft on Azure side, not only you had to pay money to find tune the model, but once the model is fine tuned, you had to also pay Microsoft money or open am money to host that
fine tune model. And then on the top of that the charges for your tokens that you will be exchanging.
It sounds like rags the way to go there.
Yes, oh you know what you say that, Richard, But we have some mutual friends in Europe, and one in particular whose company decided to do away with RAG in their company because the percentage of errors that came back was too high. And I just asked this person literally two weeks ago, has your opinion changed? And no, no it hasn't. It's still still or prone. So what do you think about that?
So RAG, you know, on theory, it looks simple, but one of the hardest challenge I have seen with the RAG is let's say I'm looking for a specific information from my company data. There is a limit on how much of the day, like how much of a token size or context size you can send to a large
language model. So most challenging part with the RAG implementation is how I will make sure for that particular specific question or conversation I'm having, how I can get the least amount of data that is more relevant, like how I can get that, And that's where you know, then you have to have vector databases. That's where you need to have your ash eye search and those are like configuring those and getting those right are easier said than done.
And let's say if you're let's say, if you the example you gave Carl is in case they're saying less errorrates, sorry, more errorrates on RAG, maybe you know they will go with fine tuning because there they have the build capacity in terms of you know, fine tune the model. Because fine tuning is not everyone's cup of tea, you know,
to have certain skills. And also they may be have a use case where the cost will justify, you know, aware or the cost the amount they had to put in the fine tuning will justify in terms of the errate they're getting on the right side.
Yeah, how much would it cost for the rag of the Franklin Brothers band to get the lead singer? Right?
I don't know.
Those are the questions you're gonna be asking, though, you know, we want we want accuracy because sometimes you can ask a question if you're if you're looking for information, right,
real information. Typically you're asking a very specific question. Yes, the answer could be yes, or it could be no, yes, or the answer could be, you know, one of a hundred possibilities, and that is usually a fundamental question that business decisions will be made on because if it wasn't, what's the point of rag.
Right.
We need to ask these fundamental questions, and then you need to base decisions on See, we have to be really, really sure that that answer is correct.
And one of these challenges I remember from you know, decades ago, where you only have to generate a report once that has inaccurate numbers, and everybody questions the numbers forever after that. Yeah, that's right, So you know, you can't afford to lose that confidence.
And that's where you know, a lot of companies, especially after the rise of Jenny I especially the big companies where if they get out any wrong information or incorrect information, if there is like a significant financial penalty against it, the first thing they are doing is cleaning up their house, like cleaning the data. Like what if your data you
have multiple versions of the data. One is saying good thing, but another saying is bad thing, and then somehow your AI search or whatever other two you're using picked up the bad information and then use that to the for you rag pattern.
I remember Gary Short and Seth Warez both saying that the majority of their job, and this was maybe ten years ago, was cleaning data, you know, just to get more accurate. And so the ability the fine tuning, as you say, to omit things that are outliers or standard deviations that are way high or something.
And we now up here in Canada we have a core precedent around this with Air Canada. So Air Canada deployed a chat bot using an LM for providing advice for travel, and a fellow used that tool to figure out if he could change a flight for it was a bereavement. Fares are called sover Yeah, a funeral or something like that. This is a great story. Yeah, And the software said, yeah, no problem, you can change that.
And so then he went to actually change it and they said, no, you can't change that, said, well, the bot told me, and then he was forced to buy the ticket the last minute spend even more so forth. But he's now gone through the courts and the court ruled, hey, that you're using that chatbot as if it was an employee. You have the same liabilities to that employee. So if your employee gave incorrect information, you'd be held accountable for it. Your software, you're also held accountable.
This is a great time to be alive in the you know, seeing the real genesis of AI taking off and getting more and more accurate in the in these kinds of stories really shape it well.
And it's I'm happy to have those case laws in place because it's for me as the architects, sitting down with leadership saying we want to use this technologies, like here is the risk we are assuming, right, right, And so you know that's just why you spend money on testing.
And that was a yes or no answer, right, It was.
A little more complicated than.
That, Can I get a refund? Yes or something along those lines or something along those lines.
And yeah, you're right. It pretty much breaks down it exactly that, Carl.
Yeah, and that's the reason. Like some of the customers I work with, they are pretty large customers, and one thing they're doing is hey, EI is great, but nobody should be launching anything because it going through some governance committee where they will be evaluating what exactly you're doing.
You know, what technology you're using, whether it's going to be for internal users or for your customers, because if you do any mistake, and let's say if it's for your outside customers, you know, the implication of that can be tremendous.
Yes, now, definitely the motion at this moment here we are, you know to a latter half of twenty twenty four is all internal apps. It's an hr bot, Yes right. It's those kinds of things where you're only upsetting the employee when you lie to them about how many vacation days they have.
And again that's a good point. You got what, Richard, because vacation days is something that you shouldn't be relying on general to AI. You tell you, it should be coming straight from your employee handbook, where where you know you have full control over what information is getting from there and what information is surfaced.
If this isn't a project that makes sense Pshan, what prout checks do like that seems like exactly, I'm not going to read the handbook. Yes, I'm going to pester HR for an answer, yes, and they're going to yell at me. So we make the bot to solve that problem and you.
Can solve it easily. Now with Copilot Studio along with Azure Ai Studio as well, but co Pilot Studio, a lot of companies are also using it for customer centric BOTE outward bots like Business to Customer as well. And one of the reasons they are doing it because some of the red teaming and other stuff the co Pilot Studio team has done because not peny people understand when you launch anything external, especially at childbot like Microsoft learn It in twenty sixteen, where people will try to get
some nasty stuff from it. And if you launch a public facing website chadbot, you know, anyone can start interacting with it and start asking some stuff that they shouldn't be asking or your company shouldn't be giving responsor to that.
So business bring us back to outward facing is not there yet. It's too high.
Risk y's high risk. Yes, but if you know, if you use the right tools, or if you know you know you have a good team in house, you can definitely use some of these tools where you can have a more confidence, you know, launching this outward facing application as well.
I just debate that anyone has a good team at this point.
What are the things that I think would help And it's definitely almost impossible with large language models that are trained on language right and try to predict the next
word in their answer, but citing references. Right, if that Air Canada bot, you know, when it said yes you can get a refund and here's a link to the policy that you can read and decide for yourself, right, then it might have been a bit more accurate, and not not necessarily accurate, but it would have allowed the user of that bot to make a more informed decision.
Yep.
And I think KYL, like Microsoft co Pilot studior team should hire KRL because now he's letting all their good
features indirectly via all these success stories. But yes, that's one of the advantage where this co pilor studio giates is if you are going with the rat pattern and let's say you're using your knowledge based then while it generate the information, it does give you information about the citation, like what are the source of this information and so that you can click on it and then you can verify Okay, this information looks great or I can then
you know, use it with more confidence and then I'll go on with the next stuff.
That's great. It's so good to hear. Yeah, ask chat GPT for a reference. Sometimes get that and on lessly. It doesn't know the way, it's bunchet. It's a very difficult thing to deal with. Yah.
Yeah, I think this is where they it's just where they use the term grounding. I don't know why this term was necessary.
Yeah, they use the grounding. Yes, Like, okay, we use this information from grounding. It's not like something the lllum made up and this is the source of it or proof of it.
Can we just always be grounded like thing? Right?
Like really like the people who have kids, you know, grounding me is totally different in their context.
Yeah right, true enough. Or the people who like to walk barefoot out in the lawn every day because it helps their magnetism or something.
Walking barefoot and grass is pretty good. Man. You don't have nice.
It's nice. It's a nice feeling, but it's all it needs to be. There's some weird science that is associated with it would say not soccer, but okay, pseudoscience anyway.
Now I appreciate that. I don't want software speculating on my van. I prefer it to say I don't know.
Yes, and those kinds of things you can configure in co pilot studio, like you can have Okay, I'm launching this copilot and it's going to take care of only four or five things. Anything extra, people like, anyone ask it can just come back with I don't know or I'm not allowed to talk about that. Go through this link or talk to this person and then and do this.
In chat GPT's defense, it does that, you know, when it doesn't have an answer, it says you should call the manufacturer or you know, contact the whatever the authority.
One thing people don't understand is chigibt is a custom experience, Like you're not interacting with an LM directly like as an API. It's a customer application. A lot of guard rails, like Opena has put into it. Now they're launching a memory as well, so a lot of things are going. It's a custom app, so yes, if you have a team that can build a custom app like that, then yes you will get similar answer. But let's say if
I'm a developer, I'm calling this LLM directly. You know most of this LM don't have a memory as well, Like the calls are stateless, so those kinds of things it may not say, and it may just give you completely like completely incorrect answer as well.
Yeah, and I guess that that brings me back to this whole. Like the HR bot makes sense is if you can ground it on. I know you're not going to read the user they employee hand books, so I'm going to turn it into the ability for you to ask questions and it's going to reference accurately. And when I'm not when my certainty is low in the SAW where let software certainty low, it's like you should talk
to HR about that. Because from an HR perspective, if I can take eighty percent of the stupid questions and just answer them so I'm only dealing with the twenty percent that are more complex, I'm pretty happy. Like that's a good outcome.
You can use for HR, you can use it for your IT support as well, because it has connectors where it can talk to your own knowledge base and that knowledge base can be in service.
Now yeah, no, I just put I have an automated responder just says you try turning it off and turning it back on again.
Like said, yeah, my favorite code is be smart restart.
You know there you go. Nice? Nice works for phones too. Yep, yes, all the time. And Pat and Dwayne Patrick Kines Douain Laflat from security this week say you should be restarting your phone at least once a week at least if not, you know, three or four times a week. Yeah, all right, So where do we go from here? Is there more that we haven't discussed? Like how is it expensive? How do we get started?
Okay, so cost wise, even though it's a part of Power Platform, its cost is a little bit different. It's just like ash. Everything is based on the consumption. So the minimum pricing point is you had to bot. You had to buy a package for twenty five thousand messages, which cost two hundred dollars. And again this is a retail price, like if you have an enterprise agreement, you may be getting some discount on that, but that brings up a question is what exactly a message is? Right?
A message is basically anything like when I ask anything to my copilot, if that's going to trigger any kind of response or any kind of AI agent a call behind the scene, that constitute as a one message. And if I'm using generative AI features, then just for the sake of simplicity, what Microsoft has done is any answer that has generative AI in it, it is just going
to count as the two messages. So if I ask a question you know how many PTOs are there, and I'm using some of the generative EI features and then that come back, then it just counts as the two messages.
Now it's two hundred dollars a month for twenty five thousand messages a month. Does that mean that if I only use one thousand messages or you know, less than that, am I only going to get charge point zero zero eight cents per message?
No? No, you have to, Like the minimum is two hundred.
That's what you have, whether you use it or not.
Yes, And that's one story which is really not good in my opinion right now, because you buy that license for per tenant. Okay, but what if I'm working for a really big organization where I have thousands and thousands of employee and ten departments are there. Every department has their own copilot and how exactly we're going to manage the pricing because those twenty five thousand message I can be exhausting in one hour or two hours.
Yeah sure, yeah, yeah, it feels like it's beta pricing, Like it's just pricing because you're doing a beta test project, not actually deployment level yet. And so now I'm starting to feel like the relationship between copile studio and as your AI studio is the same relationship between power platform and visual studio.
Correct, right, Yeah, that makes sense to me, Richard.
So if you're the front end stuff, the place to experiment, the price to prototype copilot studio when you want to get into precision and tuning and more skilled people AI studio knowing that anything you build an AI studio you can surface in copilot. Yeah, and all right, I dig this. This is huge in development. Like you said, we did that show with vishuas earlier this year where we talked about that same dynamic between power platform and dot net right.
That the and I know yeah, which Fus is a huge advocate of future development. In fact, he's the one who taught me all this concept, and I agree with it because the more and more I work with enterprise customers, like there's always the case where developers, like good developers
are a rare commodity. Yeah, and you'll agree that, you know, it's very hard to find good developers, Like how about then we create some tools where you need developers only when that tool cannot do certain things for you, right.
Right, But the other part is like I never look at the power platform folks, and I guess it's the copilot studio folks. As these are the lessers, these are the domain experts. They're actually going to save you time because they know more about the problem space than you do. Right, Like, my struggle with really great developers is how much time
they spend having to learn the domain. Where if you have a good domain expert relationship going there, you can focus more on breaking those bits down and go, okay, well that's where some custom code needs to live. That's there is no easy solution there. We need to build something, take a spike, you know, those kinds of things. Like that's a really constructive way to work quickly.
And I'm seeing like so I'm talking about this conversation AI or Microsoft Board framework since twenty seventeen and since last Ignite. I'm saying like just a huge interest from developers as well, like pro developers thinking about or are
eager to learn about co pilot studio. Like a couple of weeks before, I was at dev Intersection or next Gen EI conference in Vegas and I did session and a workshop on copilot Studio and its attendance was more than some of the popular you know, pro developer sessions or confront or workshops were there.
Yeah, that's great, but any Yeah, I appreciate this because you start now, you start thinking about rethinking the UX of an application, a lot less buttons and sliders and a lot more texts and most of the screen taken up with it presenting information to you based on your requests.
Right, And it doesn't have to be only this. It can be. Yes, for some things you'll be using this, but for some more traditional or more things where you definite need to have you like proper UI with multiple buttons and multiple sliders, yes you'll have that, but this can be your beginning point where I'm every day I'm starting my work. I'm just coming here, you know, doing my stuff. I asking copilot to do a bunch of stuff for me. And I always give the example, you know,
Javis in Iron Man. That's a good example. Hey, there are certain things if Javis can do. Let Javis handle that. And I'm going to focus more on the complicated stuff behind the scene.
Yeah, I'm waiting for office to bet on this tap. Right, Like you think about immediately think in terms of the old Alan Cooper of the Sovereign app. Right, what's the first app you go to? Now? For me? It's still Outlook, you know. That's why I'm sad.
Six five threads one for me.
There's also teams, like a lot of folks, that's their starting point. And you start thinking about the LLM interface to this, how you would start your day? I mean right away, your first step in either app is triage. Yes, right, what's the most important thing? Right? What do I need to deal with?
Now?
You know, where are the crises and so forth? Like you could see a good agentic type tooling in this, helping this or through those priorities. I mean, I would argue the first triage is how many of these has the bot already responded to?
MM?
Hmm right, I've not only sorted these out. Here's the ones I think. Here the answers are, are you okay with this? It's like yeah, yes, and that that that that that that okay? What are the ones I need a trioge like my wife could my life could be way better?
Yeah. I think that story is still evolving with Copilot for Microsoft sixty five. Uh, and they do have one capability there where in case you would like to extend the functionality of out of the box co pilot there you can use Copilot Studio and its license is already covered as part of your Microsoft Pay sixty five Copilot license, so I can listen.
Prashan, I'm an old grizzled Microsoft observer. At this point, you're not grizzled. I've come to appreciate disper he said on a podcast. You will know a technology is being taken serious by Microsoft when they bet their mark key products on it. Yes, right, when that shows up in Outlook. Okay, they're committed, like, no two ways about it. That's why we bought into Active Acts or you know that whole
control surface. That's why we bought into Ola, and it's also why we were suspect of WPF when Office never took on WPF. It wasn't until Visual Studio took out WPF. They're like, okay, right, Like that's there's a reason to take the cues this way as to how far down the path these products really are. Like I feel like these are still experimental products because the Microsoft seems to feel these things are still expound products.
Well, they may be working on them, but they haven't surfaced yet. Yeah, I have a question for you, totally off topic, well not really. Apple's coming out very soon with probably out by now, the new iPhone that has AI Apple Intelligence not artificial at all, and I kind of I feel like I get it. They're basically taking the SII experience and exposing you know, GPT four oh on the back end so that Siri will become more intelligent.
And I just wonder if you know anything about that, because I, you know, if that is as easy as it sounds. Man, I'm going to get in my car, turn on car Play, and just start talking to my car and it's going to be night rider.
You know, what do you think for Shadow? I have thoughts.
So I'm not an Apple person. I use most of the Android stuff. But after saying, like some of the demos from open Ei recently, the demo ware, the latest version of GPT four oh is multimodel, and then we can have multiple conversation and we can also change the context in between. I think we are getting there as long as they figure out how to run this LLLMS or all these models cheaply on the devices. Yeah, I think we are there, like maybe one or two years.
Once they put some SA safeguards in it. And let's say I'm driving a car and if that LM can execute on my car's hardware, we should be there.
Are you saying that the LLLM stuff is on the phone, they're not just calling an API out to GPT to do the work.
Yeah, Like once that gets mainstream, like where we will have more and more this small language model is doing all this stuff, but they're actually executing on the local stuff, we'll have like much better experience, and we'll also see more and more push from these companies as well, because.
I don't know if that's the case now. It seems to me that they're just calling out too with an API, but that.
Is a very expensive right now, Like if you look at yes, they can do it for a few months or a few years, but that's not really long sustainable business model because all these lllms require really bf infrastructure. That's the reason why I think. I don't please don't quote me on this, but I heard even though openan AA is making so much money, like billions of dollars, they're still not profitable because they had to spend a lot of money behind the scene on the infrastructure to support.
That far and far more.
Yes, what was your thoughts, Richard?
I think Apple is late to the AI game and needed to announce something, and the fact that they keep pushing back the release of it is proof they don't know what they just committed to. Ooh, and they're panicking. You know, we expect Apple to come late with a refined product that has clearly not happened. The fact that they've already pushed the dates back shows they they said they were going to jump before they jumped, and now
they're struggling because it is a very unrefined product. It is not asked Apple asque at all, but they could not afford to not say something good way to hear, so they had You know, they only do one show a year. They do that show in the WWDCS in June. Were you really going to wait till twenty twenty five? You couldn't, so you said it, and now you're trying to make it come true. That's not very apple asque,
but all the hallmarks are there. The only reason we might give a pass is because Zappli you don't do that. It looks like they did this time.
Well that feels like the end of our show, Prashant. This has been very enlightening and I appreciate you putting up with my digressions and you know, little voyages into stories and things. But man, this is just chocolate block full of great meaty goodness.
So thank you, thank you, thanks for having me. And it was really it was my pleasure, you know, pleasure on all of my side for.
This fantastic all right, and we'll talk to you next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produce by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t R O c k s dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two.
Thousand and two. And make sure you check out our sponsors. They keep us in business. Now go write some code, See you next time.
You got Jamal Vans
And