How'd you like to listen to dot net rocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month. We'll get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com.
Hey guess what. It's dot net Rocks Steve Sanderson Edition. I'm Carl Franklin, the amateur Campbell And uh, what can we say? Richard New Year?
We're you're making a show and checking it twice? Oh wait, no, Christmas is over? Yeah?
Uh all right, Well we got a lot of sort of pre show ramble to do, so let's get right into a better no framework awesome. Well I don't have a framework piece or anything per se, but I do have some news about Blazer Train and Blazer Puzzle. So Blazer Train I have done sparsely since we started the Blazer Puzzle, just focusing on redoing old things. But I've started a series on Blazer Train now that's going to be weekly or every other week on What's New and
Dot at nine. So the first one is published maybe even the second one by the time this comes out, and the Blazer Puzzle is going to be a little different than usual. Typically, what we've done with the Blazer puzzles we give a problem and we say, okay, take the week or a few days and find a solution and email us the solution. Then we pick a winner from all the right answers. And we're not doing that anymore.
We're going to give you a problem and then we're going to take a little pause, give people a chance to duck out or press the pause button on the video, and then we're going to give the solution. So no more winners, no more mugs, no more don't post this on social media. We want you to engage with us more. So it's kind of fun. It's more fun.
That's cool, man.
Yeah, So that's what I got. Just some updates from my Blazer videos. And also we should probably just briefly go over some things that happened in nineteen thirty six.
This is episode ninety thirty six.
Yeah, since it's episode nineteen thirty six and I want to be labor stuff and we don't want to take too much time. But there are a few important things that happened. Of course, you know, Nazi Germany just kind of ratcheted up in general. So there's a whole bunch of things. But on February fourth, Keynes John Maynard Keynes published The General Theory of Employment Interest in Money, a revolutionary economic work that fundamentally transformed modern modern macroeconomic thought
and policymaking. Everybody knows about Keynesy and economics now. On February nineteenth, Marion Anderson performed at the White House, significant in the context of racial segregation, would later be remembered as a pivotal, pivotal moment in the civil rights movement. March first, the first p seventeen bomber was delivered on March seventeenth, Saint Patrick's Day Flood in Pittsburgh, Terrible, terrible flood.
April thirteenth. Hitler appears on Time magazine cover. May seventh, Italian annexation of Ethiopia, and they continued to do a lot of damage in Ethiopia, the Italian military in On May twenty eighth, Alan Turing submitted the groundbreaking paper on computability. And he's a hero among us developers, of course, in mathematicians everywhere, and you know, do you have anything else you want to add? I mean, there's tons of things that happened, but sure, well you know, I stick towards
the Zion stuff. So in nineteen thirty six was the year that the last thylacine died, also known as the Tasmanian tiger. Although it was more wolf like than anything else. It had been hunted because they believed it had been hunted for bounty because they believed it was killing sheep. Turned out to not be true. The last one died of neglect in the Tasmanian zoo. This is strictly a Tasmanian animal, and coincidentally, there is an organization working hard
to de extinct the thylaccene. Wow like Jurassic Park style or.
Well, no, his Jurassic Park is fiction.
No, no, of course, but you know what I mean to cloning DNA.
But in Jurassic Park's approach was to find DNA of the dinosaurs and then fill in the gaps of amphibian DNA, which is you know, again it's kind of fictional. What they're actually doing, these folks are doing, is they are finding the closest living genetic relative because they have good genetic material for the thylacine, and then they are changing
that DNA to match the thylacine. Okay, but they're also working closely with Tasmani itself, like if they're going to does it make sense to reinduce it all that sort of thing, and there's a strong will to bring that animal back. It'll actually helps stabilize the ecosystem in Tasmania. Now that the thylacine has been gone for some ninety years, Tasmani has been stuffing air effects for not having a top tier predator, and so there's an idea that bringing back the thylascene would help.
See you get a mini geek out in every dot net Rocks show.
There you go. I mean, I've been keeping notes on de extinction for the mammoth and a few others and so forth. The thilastine has been one of them, and it just happens to be that nineteen thirty six was the year that the last thyloscene died.
Wow, well, who's talking to us today?
Richard Graudy comment off of show nineteen twenty two, the one we did with our friend Dan Roth and the Fall of twenty four talking about Blazer and dot net nine. And this comment seemed particularly relevant to our conversation today because it's from Brent from Atlanta, who said it was Grelway. Was great to hear from Dan. He binds me of a brilliant co worker of mine who just retired from Mongo dB. I'd love to know if coding llm's running locally like the Quinn two point five coder, will be
easily integrated individual studio. I'm looking to enhance my Blazer coding with such tools without having to pay for Copilot, which is preferred to be in verie to Quinn. Anyway, I love the show. Well, hey get out copilot now free, So so much for paying for that. B All the copilots now have options for different back ends you can run on, so you know, the default is the open AI one, but if you want to switch to a LAMA or cloud or anything like that, you can do that. Yeah. Absolutely so there.
Just as in a side I was talking to the guys at dev Express and they have a control now that tracks all of that stuff away and just gives you like an input box and you can connect it to wherever you want without any kind of nasty pain.
Nice. Yeah, so yeah, all good things happening, good things, and so Brent, thank you so much for your comment. Copy of music Cobuy is on its way to you. And if you'd like copy of music Cooba, I write a comment on the website at dotn at Rocks dot com or on the facebooks. We publish every show there and if you comment there, we're read it on the show. We'll send you copy of music Goba.
And you can follow us on other social media. We've been on ex Twitter forever. We're on mastadon and on blue Sky, and I believe Richard's also on threads. I have a Thread's account, but I don't watch it, and you know, but you can find us out there.
Yep, we're out there. Blue Sky seems to do the fun one of these days.
Yeah, just some variation of at Carl Franklin at rich Campbell. So let's bring back our friend Steve Sanderson. This is a very humble bio that Steve wrote for us. He's a developer on the dot net team at Microsoft. He focused as I'm making dot net better for application developers with a focus on AI and the web. Yeah, he's done a few things, a couple of things here and there, not really mentioning in the bio.
Welcome back, Steve, Hello, thank you for bringing me back.
Good to have you.
Yeah, absolutely, man, And you know I always lead this off the same way. What are you working on?
What am I working on?
Well?
I think, like quite a lot of us, I've started to focus on the AI side of software.
Oh wow, recently.
Yeah. I think it's a really big opportunity, and there's many different ways you can think about what it is. Like on one level, it's like a tool that can help you write your code, or it's just like a chat interface or things like that. I think all those things are fine, but that's not what I'm personally drawn to as like an area where we can exploit opportunities.
Maybe it's my background, but I feel really drawn to doing things around applications and doing things for application developers, and I think about the you know, the many many people I know who working companies that are producing software for their customers or for their employees, and things like how do you actually use AI within that to make better software? How can you make your software have better features than it would have had before?
Well, and when you were thinking about that, like I remember you thinking about trying to make it easier to put data on web pages, and that made knockout. Yeah. So, and then you were thinking about how to make it easier to use c sharp on web pages, and that made Blazer. So I kind of like it when you're thinking about things, Steve, good things happening.
I remember last year you did a demo at one of the Dona comps or something like that. You did a demo of some AI in AI chat wired up to data. Yeah, it was in Blazer and it was just mind blowing. So you've been doing it for a while.
Yeah, that was our e shop support demo. I'm sure we can add a link to that. Yeah. That was some exploration around this of what you get when you put Blazer and AI and typical application business app scenarios together.
And that's an exploration of like a customer support scenario where people are submitting inquiries and there's some AI that automatically classifies them as different types of thing, and then the staff who's trying to deal with customer support gets like a chat assistant that's able to look through corporate
or enterprise data to provide suggested answers. Fills it out, you know, rates the customer's satisfaction automatically, all that kind of state thing, and it just represents a whole own of scenarios that we think people would probably actually have.
Yeah, I'm looking forward to the day when I can replace my report criteria selection UI with just a chat box. Yeah, and you probably can do that already and I can actually, but you know, it's going to take a little bit of a paradigm shift for users to want to go for that perhaps, But you know, nobody's saying you can't offer it as an option.
Yeah, totally. And you mentioned chat as an interface there. I think chat is clearly something that people strongly associate with AI, and it's a thing that we didn't really have as a form of UI before AI came along, and so that's kind of exciting and cool. But I also whenever I talk about this, I really try to emphasize the fact that chat is not the only way to benefit from AI, And personally, in a lot of cases, I don't actually like typing into chat boxes like it.
When I'm using my computer to do something productive, my mind is focused on the task and I don't really want to be like explaining stuff in English to the computer. Yeah, and so I think a lot of the time when we think about how we can add AI to our applications and produce genuinely more productive apps, it's about using AI on the back end just sort of like almost
like secretly, without involving the user. It's just predicting what they want and doing parts of the job for them ahead of the time so they don't have to do it.
It does bring to mind a couple of really good text interfaces where you're writing and then things are helping you. Obviously get ub copilot and go back to intelligence and statement completion and see sharp or even writing queries in SQL management studio, you know when you say select star from and you get dropped down. You know, you could implement that in a chatbot where you could say I want to see all the reports of type and then
you know, transaction type, whatever, whatever type of reports. You know, those things will probably come in the in the AI front end chat boxes.
I'm sure they will. Yeah. And another good example of that is, of course, as you already mentioned, visual studio Copilot. And one thing that I really love about that is the fact that I don't have to explain to it what I want it to do, Like it's there while I'm writing code and producing sensible suggestions for me without me stopping to say I need to do X. It's just you know, inferring that from what I've already typed in code, which is I've.
Seen this happen in Excel. I don't know if you've played with Excel lately if you have that stuff turned on. But if if you're doing a pattern of entry into a series of a column of cells, it starts pre populating it with the next one it thinks you want, and it's I mean, the easy one is like one, two, three, but you know, first of each month, it's figured that one out, Like you're exactly right. It's not doing it
for you. You're not telling it what to do. It's just looking at what you're doing and then shortcutting instead of me having to type it out. Now I'm just like tab yep, I'll keep that, tah that and so forth. Like that's really subtle and clever.
And it doesn't have a lobotomy like Clippy. But it's the you know, tried to it's trying to solve the same problem. Really, I mean, it's the whole idea of clippy, you know, without a lobotomy.
Well, when we've had smart populating and forms and things like our browsers know our addresses and so if you looks like you're starting a type of address, it finds the rest of the boxes and fills them all in for you. Like that kind of thing, and actually address fill in is one of those ones I think people have worked really hard on where it's like, hey, when you seeing your postcode, okay, well now I know your city in state right, Like.
The country and postcode sort of reverse.
The thing up. But now I start thinking about forms over data, Like I'm really interested in the machine learning models that study a user's workflow and start to tailor their behavior to that user. Like if you do the same thing every day in this app, how do I make it quicker for you?
Yeah, that's an interesting area. People have started speculating like about really far off futuristic things which are interesting. I don't know what the sort of short term thing is here, Like there's already products and demos out there of AI systems that generate a UI on demand, so you know, while a user is doing some task, there isn't a pre defined set of textboxers, a pre defined set of wizard steps or whatever. But some AI is being asked to make up a UI on the fly based on
what the user appears to be doing. That's tailored towards that task, And I think that's pretty much still a sort of proof of concept level right now. I doubt that anyone's completely relying on that to ship and app with, but it's conceivable that, you know, five years from now, the way we interact with computers will be much more sort of generated on demand for us rather than pre defined.
Is that one of a good example of what you mean by AI working on the back end without you know, you really noticing it all that much, or were you thinking of something else.
I was thinking a much simpler thing. So the dynamically make up a UI as the user is working is like super sci fi futuristic stuff, which by which I mean like it's not going to be several months away.
But like you know with the pets of AI.
But no, what I meant was something like really straightforward stuff that you can code up in in ten minutes. So the kind of like core features that you can do with AI, stuff like being able to automatically classify text, automatically extracting structured data from fuzzy inputs, whether that's plain text or images, ability to summarize, translate, ability to detect
anomalies in data, the ability to score user sentiment. All these kinds of things you can do programmatically on the back end of your application and then use that to you know, trigger workflows automatically. Like some message comes in and it appears to be a complaint, Well you can automatically, you know, send a message to the person who deals
with complaints or something like that. Or you get something that you know, you get a large amount of text in from something and you can summarize it down to one sentence to display as like the heading within a table or something like that. You know, there's so many ways that you can use AI systems programmatically to make your application just feel that bit more intelligent.
It sounds like pattern matching on steroids.
Yeah, in in some ways, you.
Know the things that we loved pattern matching for.
Well, arguably at the lowest level, that's what language models are.
Yeah. Yeah, it is all pattern managing matching tokens effectively, but.
It's not stuff that you necessarily have to do in raw code. You know, absolutely the language has a pattern matching feature and you're using that.
Yeah. Yeah, there are tons of application features that people could be using quite easily and cheaply, especially in terms of the amount of developer effort. Something like semantic search, for example, Like the naming makes it sound kind of fancy and advanced or something like that, but it's really not.
You know, you could you can take any search feature that you've got today and upgrade it to be semantic search, like within an afternoon, and at least at a sort of prototype level, and see whether or not that's going to benefit your application, so that users don't need to type things correctly and they don't need to phrase things in a particular way, but it just sort of smartly works out what they're trying to search for.
Well. Semantic search to me seems like one of those buzzword phrases of a promise of a better future, right, I mean, we've been arguing about semantic search since the web was invented, right. Even Tim Berner's lead was like, this is what this is all about. It's just a question of or we finally got and we finally had a set of tools that can do it consistently take the semantical information around it and say you make a more informed search.
Well, I personally think that when it comes to just using embedding models, which are one of the two main types of language models that people use today, to take things like document titles or text from documents, convert them into embeddings and then be able to search through those in a way that matches on meaning rather than exact string matches is a very straightforward thing to achieve, Like, it's a really well trodden path by now, We've got so many cheap ways of doing it that I think
that most people who have got any kind of application and do as any kind of search feature ought to be thinking about doing that.
We think about it this way, like you make a document a PDF document of rules that your software must follow, right, and then just set the AI on it and say, Okay, don't let me break the rules in my software. Okay, there's a there's a sci fi for you.
Yeah, it's it's a scary thing. So, I mean, AI has so much promise, right, But at the same time, it also kind of violates many of the assumptions that we've had about how software should work, like, particularly when you talk about rules that can't be violated or anything to do with security. You know, as soon as you bring AI into it and have a language model making decisions about what a user kind of can't do, everything
becomes so fuzzy. Like you tell the language model a certain user should or shouldn't be allowed to do something, but if that user says that they want to do it persuasively enough, then it might just let them do it anyway.
Persuading software, I said in air quotes.
But what I would do in that situation is I would want a tool that says, Okay, here's the document of rules and business rules. Right, not necessarily you know, thou shalt not or whatever, but and then say, and here's my code and here's my rules. Now generate some code from me that I can where I can plug the holes. Yeah, okay, you know, and then I can look at I can do that once, and then I can look at that code those things and see if they if that's a good idea to implement them. But
you know, that's how I want to use AI. I wanted to either generate some code or give me some suggestions and they'll let me write the final you know, do the final cut and paste or whatever and testing.
Yeah, I think that's reasonable. So that's using GAI as a coding assistant, which is definitely a very very valuable part of the whole thing. And a lot of people are doing that and that's great. There are limitations to that, you know, one of them is ultimately it might still get it wrong and you may or may not actually spark the fact that it's got it wrong. Like, there could still be bugs, especially subtle security bugs. It's not magic.
And secondly, if you're constrained to only using GAI at development time and not at run time, then that constrains the sorts of features that you can ship. So I don't think it's like the ultimate or only solution, but it's definitely a valuable and valid way of thinking about things in a lot of cases.
Right, And I you know, just as an example, what I talked about is just an evolution of what we're already doing now with you know, Copilot and all of that stuff. Yeah, so I can see those things happening, you know, more discrete tools for helping us shore up our code. Static code analysis another great way to use AI.
But yeah, I would, I would be skeptical to Here's here's something that Jeff Fritz and I were thinking about, and it's not necessarily AI, but it's calling translation services on the fly so that somebody you don't have to actually do different language versions of your application. You could just you know, you're in Japan, you bring up everything in Japanese and it goes and does a translation. But now you're relying on that translation to be accurate in
real time. Not only that, but you're racking up all the API calls that you're doing. So another idea would be to just take a tool that would generate all the text and all the resources in every language and then boom, now you've got some Now you can actually test it.
And I'm sure and a lot of people are doing that absolutely.
Yeah. Yeah. The language tokenisation strategy that llm's brought to the table made translation.
A lot easier, yeah, and more accurate.
And it does seem to be more accurate as well.
Yeah. Well, modern large language models entirely come out of machine translation. That's where this whole thing started, you know, building mL models that could translate between human languages, and the way that that was very successful through transformers became you know what we have today.
Sort of the key ingredient to what began the path towards GPT was it's the translation problem first, and it's just yeah, it's one of its superpowers. And what I like about it is, for the most part, we haven't noticed. It's just that every all translations seem to be better and cheaper and more prevalent, just showing up in more places.
Yeah.
Absolute.
In fact, it was just today that I don't know if you saw VLC the media player. The oper was media Player as a shipped to feature this morning, which adds automatic subtitles to what you're watching, is controversial because people are not sure if the quality is good enough. But yeah, you're right, it is showing up all over the place.
So how likely are you to use a chat bot on a website that pretends to be a representative of the company. Oh?
Usually, I mean I'm usually my general feeling when encountering something like that is oh no, what have they done? But I did have one example of a time quite recently where I had to use a chat butt and it actually worked, and I was amazed, Wow, I had made a booking with like Expedia or something. I think it was Expedia, and I couldn't find my booking number anywhere. Like I went to every single email, there was no booking number in anything I could find. I tried to
contact customer services. It was AI. I thought, oh, this is stupid, this is never going to work. And I asked it the question and it said, like what's your name and where you're going and stuff, and then it just immediately came back and said, this is your booking number, and it was awesome and it was correct. Yeah of course, yeah wow.
And from there, like you then searched on that or you know, opened something up like validated it elsewhere.
Yeah, well it took it gave me a link to something that I hadn't been able to find previously.
Do you do you think that chat bots should be required to tell you that they are a bot and you're not actually speaking to a real person.
I don't know. I mean within context, it can be really obvious. Like if if it's literally a chat you I and there's a little icon of a robot and it says like assistant bot is his name, then you're not going to think it's a person. Yeah, it would be a terrible person.
I've asked bots before and they have said, no, I'm not a bot, but I knew it was.
And you asked him to respond to you and iamic pentameter.
Yeah, and then you do the the voidkamp test or something.
Is that Okay, well I'm interested in that, but we probably shouldn't go down that rabbit.
Yeah, definitely trouble. I am thinking in terms of how we use generative AI models to understand the user's interaction with the app. I've been doing some work with some neuroscience companies and they do a lot of additional instrumentation on people. And one of the things they've said is like, facial expressions are not a good way to measure someone's response to things, and they're very cultural, they're very personal
individuals and so forth. But if you take that camera and zoom in on a point of skin close enough, you can actually see heart rate, and heart rate is neutral. Like the fact that now you can measure that person's heart rate relative to one step to another inside of a piece of software and say is a heart rate increasing or decreasing and that is a better indicator of frustration or But.
You have to have a baseline though you have to know it when they're not excited.
Yeah, yeah, I think it's very personal, right that you could put someone through some paces and their cases. They do initial testing for a kind of surveying, and so they sort of look at what your measurements look like on that and then go into more important ones and see right through reaction.
Yeah, well that's interesting. It possibly gets us into some ethical and maybe even legal issues. Like so, you know a lot of recruitment takes place through AI these days, and people are often when applying for jobs as to do video interviews that kind of like an AI video AI assessed video interview. You know, you could it would be really straightforward then to be using a technique like that to try and assess do we think this person's
really being honest about what they're saying on that? But if you're going to make decisions about someone on that basis, then you know, as a software developer, engineer and manager thinking about implement feature like that, you've got to think very carefully about how reliable you can make this and what the ethics of doing that really are.
Because humans have trigger words, right that may not have anything to do with what you're talking about But you use a word or something and some of you know, the heart rate will go up and they'll start sweating. But has nothing to do with the context.
Well, we've all interviewed developers. It turns out that doing being interviewed as a developer increases your heart rate. Yeah, they're not comfortable, the vast majority of the market. It's almost strange to find someone who is comfortable.
Yeah, that's even more suspicious.
Yeah, what's wrong with that, psychopath? Why are you happy being interviewed? There's something wrong here.
There's no laughing in development, laughing in interviews.
Nothing's fun. This is not fun.
We'll be right back with more with Steve Sanderson after this. Did you know that you can work with AWS directly from your ide AWS provides toolkits for visual studio, visual studio, code, and jet brains rider Learn more at AWS dot Amazon dot com, slash net slash tools. And we're back. It's dot net Rocks. I'm Carl Franklin, that's Richard Campbell hey, and that is mister Blazer himself, Steve Sanderson. We're talking about AI, his latest passion. We kind of stopped and uh,
let's open up the floor for a new topic. What's what's something else that you're thinking about AI wise.
Steve Well, I guess the main area is what do we actually do practically in order to give better tools and capabilities to dot net developers. You know, I'm on the dot net team. That's that's what my jobs. And so the main thing that we're looking at doing at the moment is producing a new set of standard abstractions for AI features in dot Net. This is a pattern that you know we've we've used in all sorts of other things. So you know, we've got packages like Microsoft
Extensions Dependency Injection. You'll know we've got Microsoft Extensions Logging and all sorts of other Microsoft Extensions things that contain standard representations of common features. So we're following that same pattern.
We're going to have already shipped a preview of a package called Microsoft Extensions AI, and that contains some new standard representations for things like language models which we call i CHET client and embedding models or I Embedding Generator as we call it, and all kinds of other standard types and helpers that you can use when you want to add some AI features into your application. And then there are implementations for all of these with many different
implementations provided. So we've got the obvious ones like open AI as your open AI. We've got ones for Olama, there's ones for Gemini, ones for Anthropic. You know, all kinds of different implementations.
So both cloudy and local implementations of LLMS.
YEP absolutely and then all from us as well. The intention is that this is a shared community effort, so we've been working with external projects like the Alama Sharp project and others to make sure that they've all got implementations of these same interfaces. And the payoff for all this for dot net developers is that you have one common program model that you can use whichever AI back
end you want to work with. And it's hopefully quite a well thought out one because you know, we've had people dedicated to thinking about this for months and it's very flexible and you can plug in your own behaviors at different stages in the pipeline and combine things and you know, do all the stuff that you will want to do, and yeah, we want to be able to build an ecosystem around that.
Are all these providers committed to adhering to the interfaces that you have or is it up to you doing the implementation to adjust if their APIs change.
So yeah, So let's take an example. So we'll say the Alarma on for example. If people don't know, Olama is some software that you can run locally on your developer machine as a way of using things like language models locally. You probably wouldn't use it in production, but it's great for development, Okay. So there are there are a bunch of different client packages for that for dot net.
So Alarma sharp is the most well known one, and you can use it's APIs directly, just like it's straight concrete APIs a newer Alarma client or whatever it is, and use it directly. But then that code's not interoperable with anything else. So what you can do is use the iChat client implementation that it provides. So you would say something like Newolarma client dot dot, and then at the end you put dot as chat client and that gives you back and in an object that implements this
iChat client interface. And now you can use that in the same way as every other AI service.
So it's really Olama's responsibility to implement Yeah, iChat client, it is totally, which is the dot Net interface.
Yeah, that's right, in the same way that Sarah log implements IE logging. And you know, that's what the whole point of having these standard interfaces is that everyone can make them work in whatever way makes sense to them.
Yeah, and I'm wondering, like, how would that evolve.
Well, yeah, so it evolves in that the industry itself is evolving and we're getting new types of AI services coming out. A good example of that at the moment are the real time APIs. So at the moment, most of our interactions with language models are a sort of request response pattern where we send a bunch of text to them and they come back with a bunch of text.
But there's a different paradigm that's emerging called real time at the moment, where it's a bidirectional communication channel a bit like a web socket or you know, just straight TCP socket connection where you can just sort of fire a load of bites across the wire to the AI system and it can send bites back to you at any time in any order, like overlapping or whatever, and those bites could represent text, or they could be audio
data or images or whatever in either direction. So that's the real time API, and our standard abstraction I chat client doesn't represent that today. It represents the request responds text based pattern. So, as you alluded to, we have to evolve as time goes on. We will add further abstractions to represent other AI patterns when they become common across multiple providers.
And these providers keep contexts too, right well, they can do so that they can.
The classic way of interacting with the language model is stateless. So if you're having a conversation in a chatbot where you say A and it says B, and U say C and it says D. When you sen want to send the message E, you don't just send E to it. You also send abcd like that because it hasn't remembered any of the conversation, so you literally have to repeat the entire conversation. Now, the app developer doesn't think about
that now, because that's like inside the libraries. But that's what's happening on the wire.
Okay, So we've been spoiled by chat GPT. In other words, it's it's doing that context stuff under the hood and we don't we don't have to worry about it.
Yeah, the WebUI does that and to be honest, they probably do some service side caching of the conversations as well, but that's you know, that would be specific to them.
Yeah, okay, interesting I've been I've got an open AI interface into my Home Assistant implementation, and essentially what it does is prefixes whatever my request is with the model of the house as part of the prompt. So it's essentially saying, here's what's in the house, now, what would you do with this sentence? And then it comes back with us, you know, so I can say things like turn on all the outdoor lights, turn off the lights in the garage, and turn off the lights in the garage,
and it will change those statements. It really translates that into it's this switch that's which they know does all the things. Yeah, but it is literally manufacturing the context each time you make a request.
Yeah, that is right.
Yeah.
When's Alexa going to catch up to this?
Hmmm?
Yeah. I don't envy those folks. The yellms have sort of caught them with their pants down, both the Google Home folks and they and the Amazon folks. They've got you respond right.
But they didn't They wanted to kill the product, didn't They.
Well, the product was losing money, losing lots of money, like the you know, the purpose behind those the Amazon device was to sell stuff on Amazon Sales tool, except that nobody did that right because it correctly became a joke about walking into the room and ordering you know, a ton of soap on someone, right like or having your children order stuff because they saw you do it. So nobody did that part. And they sold all that
gear for cost. So it was a few years ago they said, hey, we're cutting these teams way back, like we spent billions on them.
Has anybody heard about anything that they're working on in the back end?
They clearly are. Yeah, I mean that they haven't.
They have to be.
They have to be. The same time, they had just cut their teams and within months of Amazon saying that Google did the same thing for their devices as well.
Well. That that brings me to the next question, Steve, do you have an iPhone? I do, yep. And what do you think of Apple Intelligence?
Oh, I don't have one that's that new, you say, I.
Do, and I don't think it's very intelligent. I would thought I was going to get an upgrade to sirikay that hasn't happened. And I have, you know, my Apple car play, so anytime I ask it something that's Siri ye, and I'll ask it stuff and it's just as dumb as it was before. Yeah, so I don't see Apple Intelligence.
I mean, I'd be happy with even if the basic SERI worked. Like I tell my phone to like set a reminder or something and it just gives me a spinner and that's it.
Wow.
So yeah, this is very unusual of Apple. But the best guess that the Apple you know, Digerati have said is that Apple was afraid last year if they didn't say something about AI at WWDC because they only do one show here, if they waited till this year would be too late, so they basically pre aound something like normally, Apple already has all this stuff worked out, talks about it in the show and its releases on schedule. But
that is not what happened with Apple Intelligence. They came up with a term, they pitched it at the show, and then they immediately pushed it back.
Yeah, No, that was weird.
I don't even know if it's shipped globally. Yeah, I don't know if it's available in the UK. I haven't heard anyone talk about it.
Yeah, it's it is. I think you're seeing the closest thing to Apple being scared that they're missing the boat on this thing and they're behaving oddly because we would expect them. And again, I don't use Apple products, but I've lived alongside them long enough to know if Apple says they're going to do something, they generally just do it and have a good plan. And that's not happening here. Yeah, it's an odd time, but I do think we're all feeling around for the right thing to do in this space.
Like I'm excited that you guys are building tools because that gives me a standard way to interact with it in my apps. No, like for me as a dot net developer. Thanks like the extensions all I understand Right when I need this, I go, I go to new Get, I bring it in in and off I go. So it's a way to sort of create a common approach to integrating stuff into yourself. Yeah, absolutely, So. Just I grab the blog post that I'll put in the show notes because it does give you sort of a walk
through of this. And it's just that you talked about open AI. You talked about, was it.
Lama O Lama O Lama olama, And.
Then there's the inference engine.
The inference I'm not sure.
As you as your AI inference Yeah.
Oh right, yeah, sorry, as your AI inference. Yeah, that's right. That's that's almost like the cloud version of OLAMA. That's a very simple mental model for it. It's a way of running any kind of model that you want to provide from us or once from a standard catalog in a hosted service, as opposed to say, like as your open air, which is literally just open AI models and doesn't include other things like the LAMA models.
Because at the same time, as you AI has opened up to many more models, and just open AI you have all theseries.
That's correct, I.
Mentioned to the listener at the beginning of the show.
Yeah, yeah, I think in a lot of cases that's going to be a much easier way of running a wider range of models in production than you know, setting up your own like containers and serving model output from that. That's quite hard business to do because you know, these things only work well if they're running on the right kind of GPU hardware, and that can be expensive, and you know, you really don't want to be managing all of that stuff.
If you can avoid that stuff is plumbing, right. You just want to be able to send your your thing out over an API and get a useful response back. Yeah, done, it's just another API. No magic. Please just let my API, that my API call work exactly. You're creating a cloud dependency. But that's not that weird. Like there seems to be a big push to run more of these models on prem.
While we're recording this, I think C is still going on the Consumer Electronic Show, and Nvidia made this announcement about this three thousand dollars supercomputer running what they called the Grace Blackwell chip set. But they're saying, for three thousand bucks, you will run a two hundred billion parameter model on prem.
Oh it's funny, isn't it the way this competition works. So yeah, the current status quo is that you use a combination of Nvideo provided hardware and then the software from open A or another tech giant, Microsoft or Google. And so we've got Nvidea trying to commoditize the tech the other tech giants the model providers by saying, oh no, you just need our hardware. You don't need any cloud.
And then you've got the tech giants trying to commoditize n video by saying, oh, we've got our own custom silicon, you don't need Nvidia. Yeah.
Fun well, happily buying each other's products too.
Yeah, see who's going to win.
This is co oppetition, But it did what impressed me, Like, GPT three is one hundred and seventy five billion parameters. So in a matter of four years, we've gone from a cloud problem that Mark Rosinovich talked about building the fifth largest supercomputer in the world to build that model back in the day, to something for three grand I could sit on my desk. In theory it was CES, which is basically a concept show.
Right, and the stock continues to get by them. Yeah.
Yeah, it's amazing enough us that grows. In fact, just yesterday I was experimenting with what's the biggest language model that I can train on my laptop in like one minute? I thought it was going to be nothing like I thought. You know, it takes months of GPU time to make a sensible language model, Like in one minute, it's not
even going to produce words that you can read. Sure, it turns out you can produce something sort of if you take a GPT that sorry, GPT two, an implementation of that on a single GPU on my laptop, I can get something that produces pretty intelligible sentences within one minute of training.
Wo.
So yeah, it's crazy how how quickly we've come on from you know, it being a you know, industry defining problem to produce anything like the English language to being really quite straightforward.
You know, they're used to be an XKCD graphic about identifying birds in a picture, being needing a team in years of work, and now we expect our phone when we take a photo, like I've got the new Android phone, the Pixel nine, and one of its features in Gemini is when you take a photo, it writes a caption of what's in the book, and that's included, Like just prettful?
Is that on device that.
The apparently it's on device. I suspect there's a round trip to the cloud involved. Okay, pretty sure?
What I what I want to do when I keep threatening my wife, I'm going to do this is to take the ten thousand photos that have accumulated since I started saving them and upload them to a cloud service and have it identify what's in the photos, and then create a database that I can search so that I can say, you know, a picture of Richard Campbell with a glass of whiskey, and nine thousand of those ten thousand photos come up and.
List Yeah, it's just in state from me.
Yeah. I think that's another little science fiction any thing too. But I mean you can sort of do that now.
But oh you can totally do that now.
Yeah, maybe not.
You could even just have it generate those pictures even if they didn't exist.
Well, there you go.
There is that. Yeah, you don't have enough pictures me holding a glass of whiskey. You can make them.
By the way, I still haven't been able to send you a Christmas present because your mail system is borked. Richard.
Well, we had a six week strike, yeah, and it kind of made a mess of absolutely Ever.
Is it back in business now or yeah.
The strike has been stopped till the spring, so a window now, Okay, that's right done. They never resolved anything. They just kicked it down the road to this.
Steve, do you have your hands in Blazer much these days?
Yeah?
So definitely not as much as before, because you know, I've my focus is broadened and spread out. But yes, I'm still very involved in the Blazer team. You know, I've probably liked ten Blazer meetings a week and yeah, so that we're working towards clarifying a dot net ten plan. At the moment for Blazer, we've got a whole bunch of meetings going on around that.
It's crazy to think about. Dot net ten just gives me chills. Man. Yeah, I guess it's only a couple months ago. You ship dot net nine, So I guess that's only fair. There's going to be another one.
Yeah, I know what you mean. It's a bit like browser versions, like you know, Chrome one hundred and fifty five or whatever by now. Yeah, time goes on. So yeah, the Blazer team is We're actually all gathering in person next week, which is going to be fun to see everyone, and we will try and dig in to a lot of this planning and working out what we want. I think one of the big themes that we'll focus on in dot net ten is the most core scenario of Blazer usage, which is Blazer server and what we can
do to make that as bulletproof as possible. Like one of the key things that people have struggled with Blazer server from the beginning is the fact that you have to have this constant connection. And yes, we've always had a sort of reconnection mechanism, but it's always been a little bit you know, suboptimal and like does it really work, And even when it works, it's a little bit like jarring to the end user and can result in them
losing some state. So what can we do to make it as practical as possible for a Blazer app developer to have a really bulletproof experience for the user so that whether or not the connection is lost, the user does not lose state.
Well, you certainly laid the foundation really well in dot at nine. I mean even right out of the box, the experience is, yeah, your code kind of freezes for a bit, Yeah, but it usually comes back.
Yeah, Yeah, that's right. We did improve it a lot in nine. In prior to nine, it used to be a pretty awful user experience, if I'm honest, where it would just sort of make it the user's problem to work out whether to reload. You know, it wasn't great. And in nine it's a lot more prone to like figure out the problem and solve it. Itself, but that's
still not bulletproof. You can still lose data because it's stored in the server memory, and if the server's gone, well it's gone, Like what else do you expect to happen? So we want to create mechanisms by which you can persist that state in a way that's independent of maintaining the connection to the server.
At nine, there are also laser improvements for auto mode. In general, it didn't really work all that well for me before in dot Net eight and before, and in nine it seems to really work well.
Yeah, so when an eight first shift, we unfortunately had quite a bad bug with the automotive that basically meant it was never going to work, which was terrible, and we patched it almost straight away, but are still a lot of people got this very bad first impression of that, and even then it was you know, still had constraints and limitations, and so yeah, we really wanted to improve that a lot in nine, make sure that not only did it really work this time, but it actually works
well in practical situations.
And that seems to be with along with some state management and you know, shifting state from server to client when we go to web assembly. That kind of stuff seems to be a really good pattern. I think now that I would actually recommend it, you know, because you get the startup snappiness of server, but then you get the you know, the independence of web assembly.
I absolutely agree with you that if your application is built in the right way and the developer knows what they're doing, then you can make it work really great. And that's that's excellent. But we kind of want to raise the bar from that from Dottnut time. We want to make it so that you don't have to be someone who writes your code in a sort of elaborate and carefully thought out manner. You don't have to be an expert, but rather just the natural way of building
your code in a simple, obvious manner. We'll just have all those benefits just innately. That's really how we want to raise the bar and improve things.
Right, so we can still use server, because using server is just by itself a huge benefit because you don't have to have an API layer and you don't have to extra security precautions and things like that.
Yeah, it's very very straightforwards. I really enjoy whenever I get to write a bit of playser server code because it just bypasses so many of the boring boilerplate tasks that you have in web development and just means you just get something running very quickly and it just works.
Well, I'm excited. Let me know when I'll have FILS.
A long way from those experiments you showed in twenty sixteen. Friend, my goodness.
Oh yeah, yeah, yes, indeed.
Yeah.
Is that the best talk you ever gave?
Or well, it's definitely fun. It was really cool.
Actually to do that, you hurt David Fowler's head and that's not easy to do. Yeah, yeah, that's true.
Yeah, very cool.
It was a great time. It's good. It was cool to be in the room. Definitely, moment, definitely.
Well, I'm looking forward to it, and I guess you know the repos and open books so you can just go check the progress and are the early bits out there yet for.
At ten, No, not at all, because we haven't even decided what's going to go into it. Sometimes we start our planning a little bit earlier, but we were intentional as a team to wait a little bit longer. I'd let things settle a little bit more, be more thoughtful about collecting feedback and about like creating clear consensus about what would be wanted in this particular release. So yeah, that's why it's it's brought us into January before we've got a nailed down plan.
Excellent, awesome, Well we're coming to the end here. Is there anything else you want to talk about before we wrap it up?
Not specifically for me, all right?
Best place in your neighborhood for fish and chips?
Fish and chips? Wow, I don't know. Probably got a couple of local chippies, but yeah, it's not really my thing, not really.
Okay.
You in the South of England is that fish and chip country?
It is?
It is you eat?
You must be going for a chicken teaka masala then or something.
Better it's more for you, yeah, for sure.
And where's your where's your favorite place in Bristol?
Oh, in Bristol, We've got a really good one called Urban tand or just a good tip for the for the locals. Yeah, that's that's sort of classic in my area.
Well, Steve, it's always great to talk to you, of course, and we'll be speaking to you more, you know, when things around dot ten get awesome. Ye okay, thanks again and we'll talk to you next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two.
Thousand and two. And make sure you check out our sponsors. They keep us in business. Now, go write some code. See you next time.
You got javans
That means home, then my Texas in line d