Mobile, Augmented Reality, and AI with Chris Sells - podcast episode cover

Mobile, Augmented Reality, and AI with Chris Sells

Oct 16, 202459 min
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

What has Chris Sells been up to? Carl and Richard chat with Chris Sells, the guest on episode 10 back in 2002, about how his career continues to evolve. Chris talks about working at Google on Flutter, the mobile dev stack - before departing for Meta to work on the tooling for augmented reality. The conversation digs into how AR appears to be the logical evolution of mobile but has been completely overwhelmed by artificial intelligence. Chris has left Meta to work on AI technologies and sees huge potential in making better applications than ever before!

Transcript

Speaker 1

How'd you like to listen to dot NetRocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month. We'll get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, welcome back to dot net rocks. I'm Carl Franklin at Amateur gam and this is Show nineteen hundred and twenty.

There you go, and Chris Sells we can see him, you can't see him, but he just shook his head like, how did that happen? He was at episode ten. Well, anyway, nineteen twenty was a leap year starting on Thursday of the Gregorian calendar, and a leap year starting on Wednesday

of the Julian calendar. The nineteen hundred and twentieth year of the Common era and Anno Dominie designations, the nine hundred and twentieth year of the second millennium, the twentieth year of the twentieth century, in the first year of the nineteen twenties decade, of course, don't you feel old?

Speaker 2

Yes? It's also when the League of Nations got formed.

Speaker 1

Ah, are there any other trivial details you want to say about nineteen twenty Richard.

Speaker 2

Because women got the vote in the United States.

Speaker 1

You're amazing. We didn't even practice this. No, he just looked up to the sky. Yeah, here's a factoid. Pretty sure it's nineteen twenty. Yeah, pretty sure? Wow, amazing. All right, As we said, Chris Sells is coming up here. I know you're interested and anxious to hear from him. But first we have this little thing called better No Framework.

Speaker 2

Awesome, bopart man, what do you got?

Speaker 1

Simon Cropps sent me this. It's not his but GitHub dot com. Slash Auto dark Mode h slash Windows Auto night Mode.

Speaker 2

So night mode versus dark mode.

Speaker 1

Well yeah, so the whole idea is that it goes to dark mode at night, right, and then and then you know, light mode in the daytime. Now, I work in a room with no windows, right, I am dark mode all the time, and I like it that way. Yeah, but I know there are people that you know, do work out in the daylight, such as you Richard.

Speaker 2

Yeah, I got natural light coming in.

Speaker 1

Yeah, and you might you know, you get this feature in phones, right, it goes to dark mode automatically. Why not? Why not for Windows?

Speaker 2

Yeah? Why should it be in your PC?

Speaker 1

Yeah?

Speaker 2

Absolutely no, that's a good idea. Yeah.

Speaker 1

So if that's the kind of thing you like, there you go, know it, learn it. Love at nineteen twenty dot po dot me awesome. Who's talking to us today?

Speaker 3

Richard gravity Komatova Show eighteen ninety eight, the one we did with our friend Sean Wildermouth earlier this year twenty twenty four on being a senior software developer, which I would argue kicked off the largest chains of conversation so far this year.

Speaker 1

And isn't it interesting that he was like a little concerned that it might not go over well and.

Speaker 2

It's like this is not an important show. It's like, I think you're wrong.

Speaker 1

Yeah.

Speaker 2

And he also didn't know that.

Speaker 3

I had Aaron Erickson coming on the following week, which the aromatic developer, same age, a little.

Speaker 2

Different career sto story.

Speaker 3

So I thought it was a good comparison and just ability to tell that show anyway. I thought one of these comments would fit really nicely with where we're going to end up going, clearly being three old guys talking.

Speaker 2

On a show.

Speaker 3

This is from Smitty, who said, I am about to turn sixty and have been a professional developer for thirty eight years. I just discovered this website and podcast as a resolve the show that Richard does with Throt that'd be Windows Weekly. Anyway, I saw an interview with Sean, who I knew from Plural site, and I gave it a listen and I really enjoyed it. I've gone through

this phase a couple of times. They sort of what am I doing with my career phase, which I think was where Sean was at at the time, and I'm in it again. For the last fifteen years, I've been working my way up through management and currently in upper management. The money is great and has mentioned in the episode. My value is the wisdom that I've accrued over the decades is useful in my department of forty plus developers

and directors these days. My question is when do I retire five to seven years and what do I do after that. I just don't see myself sitting in a rocker on a front porch anytime soon. Yep, yeah, I know.

Speaker 2

You know, if you stop, you're going to die, right We're sharks here. You need to keep moving.

Speaker 1

But yeah, seriously, though, if you guys are different obviously maybe Chris isn't. But if I don't write code at least, you know, several hours a week, I feel like, why am I even on this planet?

Speaker 2

Yeah? No, I know what you mean.

Speaker 3

And what I realized is like, I don't know a project per se, but I'm always writing bits of code for things.

Speaker 2

Yeah, a lot of it is home automation.

Speaker 3

These days, but it's still you know, you're always trying to solve problems with code because you.

Speaker 1

Can, because I can.

Speaker 3

Yeah, but let's face it, there's so many more things to do. I'm getting ready to build another machine, and the conversation with the local community spun into hey, would you build that machine at the high school?

Speaker 2

Right?

Speaker 3

You know, do a session with every with everyone to show walk them through building machine. I'm like, yeah, absolutely, Like these are things you could do any point in your life, that's right. Yeah, and there's more time, yeah for sure.

Speaker 1

Yeah.

Speaker 3

So Smittie, thank you so much for your comment. And a copy of music Coby is on its way to you, and if you'd like a copy of music, go buy. I write a comment on the website at dot NetRocks dot com or on the facebooks. We publish every show there, and if you comment there and every reading on the show, we'll send your copy of music Cobi.

Speaker 1

And Yeah, music to code By is still going strong. People are buying it and which is great. And I'm thinking about fixing fixing to come up with a new one. You're not just fixing to go fixing for uh in the right? Yeah for twenty two track number twenty two.

Speaker 3

You were saying an earlier show that you were listening to it and it sort of triggered some things in your head.

Speaker 1

Right, Yeah, it's like, you know, this isn't quite half bad, And you know, I was only worried that I was listening to it as music and not a background stuff, but it still did the trick. Well, let's bring on Chris Sells. Chris is an advocate for developers, a Flutter fanatic, and a consultant and applied AI and developer tools and ecosystems, now retired. He enjoys long walks on the beach and has technologies and I do not think he is looking

for another wife, right. I know that long walks on the beach thing kind of triggered that thought.

Speaker 3

Ye also the guest on episode ten, which like the definitive description of garbage collection. Yes, in two thousand and two.

Speaker 1

And episode one hundred and several episodes.

Speaker 2

After that, but more than a dozen.

Speaker 1

A lot of things have happened since episode ten.

Speaker 2

I can't believe you're at episode nineteen twenty? Are you kidding me? Now? You're into like the twentieth century, right, these are like years, And of course that's what you did, right, You went, hey, you know what happened in nineteen twenty, and you forgot prohibition, which seems like a bad idea. The ACLU was formed, which is a good idea. And I didn't know this. I'm sure Richard, you did that. Ponzi's schemes were named after a man named Charles Ponzi. Ponzi,

and he did it in nineteen twenty. Wow.

Speaker 3

I think the biggest esthetic around nineteen twenty is you're coming out of World War One and the Spanish Flu, and so culturally there was this sense of what we now call existentialism. It's like everybody knew somebody who had died, and for random reasons, the first industrialized war, this disease, we didn't understand. You know, at this point, Pastor has figured out what brewers yeast is. But that's about it. We're still twenty years away from any any treatments for

a lot of these things. You still had children that died from stepping on a nail, right, and so the fact that you had this insanity happen over the previous five eight years. This is where the.

Speaker 2

Whole flapper movement comes from. Where a lot of this stuff comes from. Was like party today because the day is lie. Hey, save this for subsequent shows. Yeah, the Roaring twenties, this is it.

Speaker 1

We have a lot of shows to get through.

Speaker 2

Yeah. So these yeah, these are the shows. These are your Roaring twenties, your shows.

Speaker 3

Yes, we're going to do It's nineteen twenty and we're going to roar through them because they have.

Speaker 2

So many good stories about this. I am happy to be your first flapper on your nineteen era of shows date.

Speaker 1

All right, Uh so what have you been up to mister Cell's retirement, you said before the show.

Speaker 2

Well, it's funny. I did the math in June, my birthday, and I have been a full time software engineer for about forty years at that point. And yes, if you do the math. I actually started with an Apple two plus my parents' basement when I was about fourteen, and they made me go to high school during the day, but all the rest of the time I was writing code, and I've been sixty five two assembler. It was not Oh, I did some of that right. I would indexed indirect

addressing for anyone. I remember really entering these strings of sixty five h two just by as numbers as folks

statements right from magazines. And so for Christmas one year, I asked my grandmother to buy me a speech synthesis card for my Apple two plus, which she had no idea what that was, but you know she I helped her track it down and I used it to read back to me the listings that I entered from the magazine to make sure that I got everything right, because you know, you drop one number instead of going eyes back and forth, you're literally hearing it say it, and

you're reading against the magata. That's right. I love it. Yeah and yeah. So a Swedish farmer was reading me back my code listings to make sure that I entered them correctly.

Speaker 1

You're not going to believe this, but I my first computer at home was a trsaight model. For I know, I'm like the young kid in the crowd right in

terms of computers. But I also bought a speech synthesis card that you could attach to the back of it, and I used it to play games with my friends, where I taught my friend how to enter in an input statement for a particular person that we knew, and then it would spit out, you know, a time warp that what happened, like you, the game was you woke up in the future, and you have a computer here, and you want to ask it what happened to you? Put in a name, and then it read back a paragraph.

And we would spend hours just writing up these crazy paragraphs, and the thing would speak it back to us. Just these are twenty rain stories.

Speaker 2

We talk about, you know, our first computers at Richard. I hope you'll tell us what yours is in a minute. But Tira Sadi model one. There you go, the Trash eighty, the Appleitude plus, right, I mean a friend of mine who had a big twenty. I mean, I think anyone listening to the show is going to think we're born in the nineteen twenties.

Speaker 3

No, yeah, I apparently the Tira Sad. I didn't learn this till years later when I got smitten with history, that the Tandy company did not think it was going to do well. So they only manufactured like two thousand of them and distributed one to each store, and that's all there was. And they just sort of sat in the corner. And I was the kid at ten, who

is this nineteen seventies? So and I went in and I could make it work, and they didn't care, right that I could make it work, then we're doing anything with it. So I had to find time messed around with a whole four k a ram, right, But it also then became a sales tool, right, like when people did get interested in wanted to buy it. The fact that the ten year old was making it sing and dance so you can do this.

Speaker 2

Yeah, they didn't realize you were just any ten year old, right, So yeah, it's still I guess it was nice for the store.

Speaker 3

But I was in a radio shack in the first place because I was buying parts for making stuff go boom. I wanted to I wanted to rock a countdown timeer because saying three to two one is too much work for me, right, I want to push a button and it counts down for me.

Speaker 1

For me. I did the same thing Richard Nixon. I've told the story before, except I was kind of a jokester. So I'd go into radio shack where they had the Tandies running, and I would press control C to break out of the basic program that was running, and I'd clear it and write my own basic program. And usually it ended up with swearing at the cost inevitably, Yeah.

Speaker 2

Ten print Carl rules twenty go to ten right, Yeah, Yeah.

Speaker 1

It was more like what's your name?

Speaker 2

A dollar?

Speaker 4

And then fu a Dallas, pretty simple man and just walk out as a jerk little kid.

Speaker 3

Anyway, so we did you were at Microsoft. There was a long gap on I'm just thinking about shows. Then we did the Leak Show issue which was thirteen thirty seven, I mean literally leaps sure, which was the Google Cloud Show, which I thought was really awesome, right, talking about dot net on Google Cloud and all the goodness that still exists there. And then we certainly fld a lot of flutterlove by the edmen on the show. Since you left Google for Facebook, what was that about?

Speaker 2

Yeah? What was what were you thinking? What was that about I'm pretty sure I phoned you at the time and go, dude, what are you doing.

Speaker 1

Well?

Speaker 2

And I am not and have never been a Facebook user, right, I was a I was not a Facebook user when not being a before being a Facebook user were cool, right, that's right, And so when I interviewed with them, actually was funny. I'm like, just to be clear, I'm not I don't use Facebook. I don't know if that's a deal break or not. And they were fine because at that point they had changed their name to Meta Meta.

So that's how I think about it. I went to work for Meta, and I went to work for Meta so I could work with my dear dear friend Don Box okay fair, yeah, yeah, And he was in Reality Lab. He was a VP there driving various AR and VR kinds of things, and so I was there doing deb tools for AR and VR developers for their well, for their so this quest head set now, right, and then you know we were working on and still I'm sure they're still working on AR hardware, right, you know, more

like glasses than goggles, right, yeah? Sure? Sure. Did you work on Horizon World? No? I did not. I was working on the other side. I was working on the platform side. That was one of the apps they were building.

Speaker 3

Yeah, so they were implementing the tooling you were building. Was what helped build the Horizon World, presumably.

Speaker 2

You would, Yeah, I mean there was a platform of things, and we had actually a bunch of teams around Meta building various things. Obviously Horizon World was one of the big clients for that platform, but a lot of building a lot of first party apps. The idea being, you know, what was the thing that developers needed to build for to build compelling VR apps? Apps? Not games? Right? The games thing they had locked down and they still do, sure, but apps or or mixed reality apps or are augmented

reality apps? Right, what do you really need?

Speaker 1

And so I thought Unity was that thing. In fact, I think you told me as much.

Speaker 2

Well, so Unity was the what they used to build games for sure, and absent experiences for uh, for virtual reality right where you own all the pixels. Unity is great for that and and Meta had a partnership. I'm sure it still does with Unity. And you know, one of the outputs from the Unity ide was, Hey, I

want to target something for the quest. But when it comes to augmented reality, you know, the the hardware you get on a pair of glasses or the hardware they were projecting or you know, uh dreaming of right, was going to be a lot smaller. It's way too small to run something big like Unity, right, so wise, yeah, so we were we were looking for something, you know, much smaller that you could you know, get the performance and and still figure and still build the experiences at

the same time inventing what those experiences would be. Right, what do people really want? What? What's the killer app if you were for for your glasses? Right?

Speaker 1

So, really, what you're talking about is an app that you can run on the glasses or the goggles or whatever that lets you build the experience on them, not on a PC.

Speaker 2

No no, no, no, no, no no, you build the experiences on a PC.

Speaker 1

Oh you do, okay, that.

Speaker 2

Target the app? Oh yeah, but you are trying to do the twolling. Yeah yeah, yeah.

Speaker 3

But if you think you need to paint a vision before you know what to build a tool. So for me, the thing I've been struggling with with AR is what is the information workers app? You know, what is the word in Excel that needs a augmented reality?

Speaker 2

Yeah? So, and of course that is that is a question. That's the killer app for me. The killer app I want. I want the terminator vision, right right. I want to walk around and just see all this extra metadata, have your reality augmented, that's right. I want you know, Oh, I'm great with faces, but terrible with names. Help me understand. As I approach this person, write what they're remind me of their name. And the last time we talked, they.

Speaker 1

Had that in the seventies. It was called LSD.

Speaker 2

That's great.

Speaker 1

It wasn't necessarily accurate information, but.

Speaker 3

If you made sure the person you're talking to was taking it too, they didn't care.

Speaker 2

Yeah, yeah, so you can think of LSD. I mean the other part the reason one of the things I cared about was I want useful information. I don't. I didn't want it to be just you know, this stream of ads shooting shouting it to you from all the places around me. Right, yep.

Speaker 3

Yeah, that's there, and therein lies the challenge, right, So what's the business model? But I do think it's a

very challenging space. Like every time I dig into industrial applications, like I've looked at some of the HoloLens applications where they're working on expensive machines and the fact that the software can exactly that augment these are the steps to do this maintenance, and then also keeps a detail video record of them doing it correctly, like when you're working on a fifty dollars million dollar machine doing a five million dollar maintenance. A five thousand dollar our headset isn't

a big deal. Now it's not. But that's very much that vertical industry, that certain verticals that makes sense. It's not a consumer product.

Speaker 2

Well it was interesting too because while I was there, we were definitely focused on the consumer space, but every one of our competitors had already transitioned to the enterprise space, right Hollo Lens was enterprise and government, and Google Glass was enterprise and and Magical Leap who knew what they were doing. And then you know, Meta was very focused on consumers, and of course, you know they always happen, right, I mean, that's just in the DNA of the company. Product's always been.

Speaker 1

And they changed their name to Meta presumably to promote this idea of the metaverse where they thought that everything was going. Did they think they would be there by now?

Speaker 2

Oh yes, for sure, And in fact, in many ways as they are. I mean they are easily the number one VR headsind right. They keep pumping out better and better quests, and the games are really great. They're a ton of fun, and they get they get better with their regeneration. But the thing that they kind of had a kind of unexpected hit in was the fitness apps.

And I understand this myself, right, I mean, exercise has traditionally been something I do not enjoy, and so if because it's just boring, right, and so the nice thing about the Quest is, you know you can do all kinds of amazing things that are actually good for you, right, You have fun, you're you're mentally engaged, and and and and so they are easily the number one in that space. And then nobody's really done the ar thing yet. I mean,

Apple's Apple what Apple Vision is, uh? What It's it's mixed reality, right, Essentially it's Google goggles, not Googles goggles, right where they own every pixel and then they use cameras to bring it in so it looks like you're augmenting the world around you.

Speaker 1

It's like Mad Quest too.

Speaker 2

Yeah, and medical the Quest does that as well, right, I mean, although you know the vastly different price point, but nobody's gotten anything close to what you really want, which is you know, os running on the glasses, right.

Speaker 3

Yeah, well, I mean you want it self contained, but there's always that debate of you know what, what's actually necessary. I remember seeing the early prototypes of the HoloLens. They called the gargoyle because which is straight William Gibson, you know the bridge quote, Because when you put that thing on, you look like a gargin painting, right, Like it's just a set of cables running down your back, like it was a monster.

Speaker 2

Well, and that was the That was one of the tensions too, which is mr is a great kind of you know, stepping stone to ar right, because it helps you build out the experiences, but you are by definition limited to those people who are willing to walk around the world with those china And.

Speaker 1

I guess that's that's what I was getting at when I asked if we were at the vision of the metaverse yet. I mean, yeah, you can put the goggles on, and you can exercise, and you can have fun and play games, but that's not where I live. I live in front of my monitor on my desktop computer, you know what I mean? Absolutely, I don't. I can't. I can't wear that thing for more than an hour at a time because it just hurts.

Speaker 5

Yeah, you know, yeah yeah, And of course that's because of the weight, and it's because, I mean, for a lot of people, although this is less true over time as the graphics get faster.

Speaker 2

Had trouble with sickness, right, and you know me, time I put it on, the first thing I did was reflexively looked for my hands right, were my hands right? And then of course after that I wanted my keyboard ring front of me, as you correctly point out, Carl, right, I mean, but just to be clear, that stuff isn't for us, right, all of our reflexes that we've built up over a lifetime. Hey, you know, Richards started in the back of a radio shack and has been building

on those same muscles ever since. This technology isn't for us, right, will die, and the next generation will take the new technology and develop their own muscles around it, and they'll grow up with with whatever it is. Now. I would argue that the VR stuff has not made it into a place where it's useful for you know, a place where you live. As you put it, Carl, that's a really good way to say it. Right. It's not the place where people live, and I don't think it ever

will be. I just don't think that that kind of bulk on your face is something that people want. If I could, yeah, but if we actually when when we get too smart glasses not if I believe it is absolutely a win. I think people will live in those for sure. Yeah.

Speaker 1

I use immersed with the quest you know about Immersed. So this is something that you connect either via USB or Bluetooth to your computer and you get your computer screen in a virtual world, and you can add multiple screens to windows and put them wherever you want. And also you get to draw these portals, which are holes in the space time fabric. Right, try a little square where your keyboard is. You can see your hands. And

so this is a really good thing. I love using it with a laptop in a hotel room if I really want to, you know, do some real work instead of you know, being confined to the screen. It's got its drawbacks, but I like it. Interesting cool program. Jeff Fritz told me about it.

Speaker 2

I really like it. I like that idea where you can you can like move your head over here and that's where the hotel TV is and you can you can carve out a spot in your in your That's a really clever way to do it, right.

Speaker 1

Yeah, the quality for watching TV or video is not there, especially in low light, but you know it's getting better. It's a good idea, good enough for me to see the keyboard and mouse.

Speaker 2

Well, and that's the it's stuff like this. That's where the reason I I thought, I the reason I was excited about the technology is I uh, because you know, you look at you know, web, the web, and it's pretty much done, right. We know how to do things on the web, and there haven't been a lot of innovations there. And before that, the desktop has been done

for a long time. And and now you look at mobile and it's done too, right, I mean, there's not a lot of innovations in terms of user experience and new ways of doing things. And so I looked at AR specifically but VR two as kind of the next generation of user experience for computers, right, I mean, a new way of doing thing. Turns out I was completely wrong, and I think everyone was. It turns out that the new way of interacting with computers is AI is generative AI and LLMS. Yeah.

Speaker 1

Before we jump into that, though, I really need to point out that Chris, for our listeners that don't know, was instrumental in the success of dot net rocks. And he may not think so, but back when bandwidth was expensive, I was asking my friends and Chris was one of them, Hey, is there anything that you can think of to help

us alleviate this bandwidth bill? And Chris was working at MSTN in Microsoft, and I think you said, I literally walked over to somebody whose name I can't remember and said, Hey, can we help out Carl with maybe like a fee on our website? And that's exactly what they did. There was an alternate feed RSS feed for the podcast on the MSDN website for a while, so thank you for that.

Speaker 2

That was Sarah Williams, my boss at MSDN, and she was always a big fan of the community and helping out the community, and she was one of the actual before she took over MSDN, she was one of the original evangelists at Microsoft and kind of started up all the programs that we know and love in terms of not just Microsoft, but how modern technology works, in terms of developer relations and developer evangelism and that whole way of doing things. Sarah Williams was one of the people

who invented all that for the Windows ecosystem. Yeah, back in the day.

Speaker 1

There's another story about that feed, the MSDN feed. I can't remember what year this was, must have been nineteen no, no, no, two thousand and three, maybe two thousand and three. So I did. I had an interview with the Guelda Kaza And this was back when you know, Linux was the enemy and all of this stuff. Microsoft had not yet embraced open source or Linux or any of.

Speaker 2

That open sources of virus.

Speaker 1

Oh yeah, it was. It was deeply embedded in the corporate culture and so they did not list that episode in their feet.

Speaker 2

It seems so silly now, right. That was the first time for us. It was like, hey, this show didn't appear.

Speaker 1

Yeah, yeah, yeah, sorry about that.

Speaker 2

Oh it's funny too. Sarah was a shield for me when I joined because probably the most popular part of my website was the Interview question section where Microsoft interview Questions, where I would just collect questions that people would ask,

these silly questions. You know. It all started with wire and into madness after that, and as soon as I joined Microsoft, somebody reached out to Stara and was like, make him take all of that stuff down off of his website now he works for us, to have him take it down or fire his ass and She's like, nope, No, that's his website, that's his personal stuff. Leave him alone, and that's not how we do things.

Speaker 1

That's right, great, great memories.

Speaker 2

All right, let's take a break.

Speaker 1

Yeah, we'll be right back after these very important messages. Hey, dot net six is going to reach the end of support this November, and now is the time to upgrade. Dot Net eight is well supported on AWS. Learn more at aws dot Amazon dot com, slash dot net and we're back. This is dot NetRocks. I'm Carl Franklin, that is Richard Campbell, and that is our friend Chris Sells.

And just before we jump back in, I got to remind you that if you don't want to hear those ads, you can get an ad free feed by becoming a five dollars month patron at Patreon dot dot net rocks dot com. Chris, before I derailed the conversation, you were talking about AI as sort of the next thing that you were getting into. But you're retired now, so how does AI sort of fit in your post workday life.

Speaker 2

Yeah, So, like I say, I retired in June, and before that, I spent a year as head of product at a little at a little startup doing generative AI and dev tools and it was, honestly, it was an opportunity I could not turn down. Steve vah of from Amazon Cloud and Google Cloud kind of drug me into it.

And the name of the company is source Graph and they build uh search product, which they have been doing for eleven years, and it's amazing and scalable and you know, millions of lines of code and hundreds of thousands of super repositories, all instantly available at your fingertips. And what they did was they took all of that search ability that because they get not only going to do like literal search and regular expression search, but they do semantic search.

They really understand your code. You can configure it so it knows about your c sharp code and your Dark code, and your Java code and your Python. It knows. It has semantic knowledge, right, it knows the types and the relationship between the types, and you can navigate in their product. And so it takes all of that semantic knowledge, and they what they did was they they built out a product called Cody and source graph Codey is an AI coding assistant that lives in your ID. It works in

all of the jet brain's IDEs. It works in visual studio code, and it does chat and it does you know commands, and it does you know autocompletions, and it does all the things you would expect, but it does it in an environment where for the enterprises, with all of those hundreds of thousands of repositories and millions of lines of code, it brings those and that understanding of that into your into your chats, right into your use

of the AI. And just I mean that's how I spent uh, Like I say, I spent about that last year of my career. And then and then in June, I was I did the math and figured out how long I'd been in in this career, and I thought, maybe it's time take a deep breath, right, step back, do something else. And uh, uh, you know, I've been super lucky. I've got gotten to do amazing things in my career. And I looked back over my career and I thought, you know, I don't want to jump back

into some you know, executive role. I don't want to jump back into some high level PM role. I've done that enough. I really miss Carl. You and I the same page. I don't write enough code in a in a week, I start to get itchy. Right, And It's been a long time where since I've had a job where writing code was was part of it, right, And so so I I look back over my career and

I thought, where was I having the most fun? Right, Well, you know, maybe I can start doing that again, right, And it was it was very much working on the Flutter team, but a bunch of the stuff that I would do evenings and weekends right where I would you know, go and give talks or be on podcasts and talk about the technology, or where I would you know, build libraries and samples and kind of you know, help fix developers uh lives, you know, more directly as opposed to, hey,

let me understand their requirements and motivate the the pms. You know that worked for me to go and blah

blah blah. Right, I mean it was more direct and in fact, I invented the routing package that that Flutter uses, and the Flutter team now owns and and is kind of their default experience, and that kind of stuff was all evenings and weekends, and I thought, maybe if I could do that again, And so I send some emails out to some of my friends on the Flutter team and now I'm up to my eyeballs building AI related

things for the Flutter and Firebase team. You know, you'd never know I was retired based on the amount of work I've been doing lately. But it's been all hands on. Last week I was in New York giving a talk on something called the Flutter AI Toolkit, which is something I've been I've been building with Google, which is all about, Hey, you want to take you know, a Flutter widget, and that is a whole AI chat, multimodal AI chat pluggable ll M out of the box support for the Google LMS.

You drop it into your app and then you hook it up to the data in your app, and you hook the output out so that you can consume it in your app, and suddenly, bang, you can take any app and just make it better by you know, this interaction with AI. And what I've seen is and I started the I mean, I I wanted to build the simplest possible demo for this talk. And so I wanted a recipe app, right, I started with It's just it's just a list of recipes, that's all it is. And

you can add them and edit them. Right, it's a crud interface over your recipes.

Speaker 1

You can scale them, obviously, this is the best thing about a recipe app.

Speaker 2

You can search them whatever. Right, And so it's just the baarest minimum thing, so classic. But then you just drop in the chat and you hook up the data and you can start seeing things like, oh, these are

my food prefences. Can you give me a verse of that recipe substituting all the things or go and you know, give me a recipe from your training set that matches my uh, you know, food preferences and you know the I'll take a picture of what's in my fridge and my cupboard, or I'll take a picture of you know, a recipe card that my grandma gave me, and you know, parselll that and give it to me in structured way and show it as a flutter widget in line with

the conversation and press a button and boom, now it's in your database and circle all of these features where you know that you get by just dropping in an ll M and having a normal conversation in a multimodal way, and just doing a little bit of input and output between the data in your app and the data in this chat, and suddenly you get an app is ten times more powerful, right because it does the ll M is implementing all of these features. That's the amazing thing.

And I like, I like, I like it from it.

Speaker 3

It's a feature extender, right, that's right, really a new ux to the way that you build software. I think it's not an app unto itself. It's just this ability to make an app way more compelling. It really kind of answer hard questions like what substitution is going to make here?

Speaker 2

Or that's right? And it's got the because it's been

trained on the entire internet. Not only right, does it have all of the stack overflow in it if I want to generate code, but it's got all of the recipes from all of the recipe sites around the Internet in it, and so I can just say give me a recipe with X, Y and Z, and it just knows yeah, right, and then you can tell it oh and you know, behind the scenes, you can do some you know, prompt engineering, so that the recipes come out in a structured Jason format so your app can parse it,

and you can add a little button that says here, bring this over to my database. Or when you're doing the output, you do a little post processing on what the user has entered to enable RAG right, right retrieval, augmented generation. Let me look up recipes from the current database and plug those in as appropriate and send those

as input. I mean, it is so amazing how a little bit of glue with this fundamental building block and suddenly you get all of these features that now is a app dep I mean, just that one feature right before llms of let me scan a recipe card for my grandmother's you know, stack to recipe cards, right, that would have been I don't know, six engineers for a year.

Speaker 1

Well, using OCR, you could do it, but you have to do a lot of logic to figure out what's what.

Speaker 2

Well, except for my grandmother hand writes all of her recipe cards, right, good luck with OCR and that whereas I can hand it to the LM and it just works. I don't have to do In fact, I don't even as an engineer building the app I don't have to do anything that's just a feature that comes along out of the box because we're using ll ms. And that's why I say, as you you know, I thought the next way that we were going to interact with computers was going to be these you know, three D interfaces

user interfaces. It turns out that no, it's a textbox. It's a textbox, right.

Speaker 3

I mean I would also argue they had the breakthrough, right, like, we're still looking for the AR breakthrough.

Speaker 2

Yeah, that's right. I think.

Speaker 3

Was the killer app. It was the breakthrough. That was a breakthrough, and and then I open the door to a bunch of other things. They're not mutually exclusive. I think AR will benefit hugely by the context sensitivity, likely next behavior, probabilistic behaviors that all these language models have, like, all of that will make that product experience better. Once we find the breakthrough that's going to make people care because we that's right.

Speaker 2

And you know, environment generation, right, I mean, let the LLM build up an environment around you. In fact, I don't know if you've seen this, they actually I think it was the DeepMind team at Google. They have got a trained a model that will generate as you move around, they will generate thirty frames a second of Doom, meaning they've trained it on Doom. It's not using the game engine, the game is not loaded or running.

Speaker 1

You just discovered a new use for all the old shopping malls that are abandoned.

Speaker 2

We can go play Doom in them.

Speaker 1

It's so cool.

Speaker 2

Yeah, I mean the potential for generative AI. And that's one of the points I covered in my talk, the core difference. Because of course we've been doing AI for long enough, decades, long enough. We've now had been through two AI winters, right is what the AI research community calls them. We've been doing it for a long time. We've benefited, right, But there's all kinds of amazing stuff we've got from AI. But genitive AI is the first

AI that will give us new content. Right was before it was like here, let me give you some content. You analyze it, and you know, you tell me you know as a cat or a dog, or you tell me you know, the next number might be a six, whatever. But this, this is the first time where you can actually give it some parameters and have it generate anything. Language like, it doesn't have to be a human language.

It can be code. It can be adjacent. It can be music, it can be anything that is kind of has a you know, a description that you can run through an ll M. It is amazing. And of course images, Carl, you bring that up right. Being able to generate you know, stable diffusion and generating images, it is a completely different way to interact with the computer.

Speaker 3

There's a challenging step for your recipe app tell me what this will look like when it's cooked.

Speaker 2

Well, actually, that's that's part of it, right, being able to say, hey, and I'll give you a picture of what this recipe is.

Speaker 3

Yeah, yeah, did I or you know, take a photo? Did I make this ride like that?

Speaker 2

Is that other thing? Being able to say no, maybe not, that's not how it's supposed to look.

Speaker 1

I like the idea of taking the recipes from the internet, plus all the comments and saying, you know, which version of this are people the most happy with?

Speaker 2

For example, yeah, well, one of the things the LM can't do is taste it for you, unfortunately, right so, but it.

Speaker 1

Can read the comments in the rate.

Speaker 2

It has definitely given me several recipes that I do not think any humans would actually like to eat.

Speaker 1

Oh yeah.

Speaker 3

The testing backbone that sort of you know, generate evaluate cycle. Say, is this going to actually come out right like that? I think we're still tweaking into that. At certainly the latest Microsoft announcement for their Wave too, there's been a lot of this, how do we make the software iterate a few more times to tune itself per request?

Speaker 2

Well, and that's the thing, right because right now we're we're doing AI in a way that is very much interactive, right, which is I go to a chat engine and I say blah, or I invoke a feature in my piece of software and it composes, you know, a chat request and goes to the LM. We're uh, we're moving to a place that is agent based, yeah, right, where you can say, hey, I want to I want Cody, which is the name of you know, source graphs AI coding assistant. I want Cody to be a member of my team.

I want to give it a badge right at my company. And I wanted to look at all of the the bugs that come in and I wanted to do that initial triage. Is this an actionable bug? Does it give me an error message? Does it tell me about expected behavior? Does it have a minimal repro can I reproduce the bug? Right?

And if not, you know, automatically kick it back and ask for additional information, or if you can reproduce it, take all of that information and then start doing like, hey, what might be causing the bug in the state of this in my codebase, and what are some suggested fixes and stick all that in it, and then even maybe being able to say, and here's a pr right, and now let me put that in the queue for the engineers to get to. That's like eighty percent of the

work and it's all drudgery. Sure, right, Why do we have engineers do that, right when we can have you know, a software agent do that for us.

Speaker 3

That could take a lot of that on that's right, certainly over on the right as side, I've been talking to folks who are doing this with tech support tickets, where the tool now is analyzing the submitted ticket and there's a bunch of if it falls in a category of automated responses, like we'll push back the responses and potentially resolve a tick on its own, but when it can't actually then goes and does data gathering to further the process so that by the time it actually lands

on a person, right, it's been worked over like that's right, yeah, and it's just so that they're working on more meaningful work, less toil, more meaningful work.

Speaker 2

Well, and it's interesting too because you know the term prompt engineer, right, got a bunch of chuckles from from people, you.

Speaker 3

Know when it's prompt engineer and crypto bro were said in the same voice, right, like that's the problem.

Speaker 2

But here's the thing, right, if you I mean this was clear. I don't know if you if you've seen what Anthropic has been doing with Claude, and you know that they have this cool feature where I think it's called artifacts, where as it generates code, it pulls those out of the stream and sticks them over into like you know, the right hand side with the code thing, and then when it gets all the code, it actually

runs it for you. When you get a preview. Somebody leaked the implementation of that, and it read like, you know, a spec that a PM would put together and hand over to an engineer and say can you build this for me? Right? The amount of detail that you get, it's like how you would talk to a human and get them to do it. Essentially, people have switched from writing programs in code to they're switching to writing them in English. Here's what I want, go and give me

a thing. And the beauty of that is everyone speaks their language, right. I means suddenly anyone that can write a precise or reasonably precise description of what they want can get it and they don't need an engineer to do it.

Speaker 3

I like LMZ on both ends, where you just write a quick set of bullet points for what you want. It turns it into fancy pros to asperson for it, who then us an LLM to strip out the fancy prose to get the bullet points.

Speaker 2

Of what you want. It's funny to say that. Yeah, it's like, I don't want to read all this summarize it for me. There have definitely been instances where I'm like, hey, I need a proposal, I need a a performance review, I need a you know, a statement of work. I need this long drawn out document right to fulfill a requirement. But you know, really I just have the data that I care about in bulleted form. You know, let me let me let the LLM do that first past, yeah,

to dress it up. Yeah yeah, yeah. And then of course you know it's never good enough, right, you have to go through and edit it. But it's easier to edit than create. So yeah, you know, well and that's the beauty, right. I mean, it's always easier to to subtract things out or edit as you say that, it is to stare at a blank's piece of paper and go Now, which how do I start this? Yeah, so you can.

Speaker 1

As soon as you talked about Doom and AR and I made the joke that, you know, there's a good way to repurpose all these closed shopping malls. It was a fun joke, but it just won't leave my brain. Like you know, AR requires real space, you know, whereas VR you can. I can see us getting close to the Ready player one kind of circular walking, you know pad that he was using in there.

Speaker 2

To Those are real things. I've been on one of those.

Speaker 1

Yeah, so, I mean I remember they were the real things years ago.

Speaker 3

But there was a game on the HoloLens where the aliens were attacking. Would they come through your walls? Oh yeah, so map your surfaces and you'd literally see a crack up here on the wall and the alien would pop.

Speaker 1

Out of it. It quest has the same thing. Now, yeah, it's really cool, but AR requiring real space and VR not so, so what's the story there, do you think? I mean? Okay, so here's a good AR app, the Pokemon Go app, right, hugely popular, requires real space.

Speaker 2

I would say that is that app is the only successful AR app.

Speaker 1

Right, Yeah, arguably because GPS anywhere, you can put these things anywhere. Yeah, but what about you know, taking an existing space and mapping AR around it.

Speaker 2

Actually, one of the best VR experiences I ever had was I was in Vegas and they had taken a two thousand square foot space, and they gave you a gun, right, a plastic gun, so you had something to pull the trigger on. They gave you the goggles, and they put the computer in a backpack on your back, right, So

you were now a soldier going into battle. And you went with a bunch of other people and you could see them in this virtual world and they would march you around through the hallways and stairways and blah blah bla in this virtual world. And the way they did it, it felt like a huge, vast space. But they were just super clever about, you know, using the two thousand square feet and reusing it. And definitely if you've got that kind of space, you can build some amazing things.

All that that said, I'm not sure I would be super excited about going up and down the stairs in an abandoned mall with goggles on my face.

Speaker 1

Yeah, as long as you can see it. Yeah, let's face it. Being in Vegas itself is augmented reality.

Speaker 2

Yes, very much so.

Speaker 3

Remember I used to play Own this World, yes, right before the game went away, and at one point I actually owned the world.

Speaker 2

He is making the world.

Speaker 3

Because I leveraged using the road trip to collect as many the whole world's cut up into squares.

Speaker 1

He was so so antisocial, sitting up in the passenger seat the whole time, not talking to any of us, just playing on this World.

Speaker 2

The hell has owned this world. I don't know, I know what you're talking.

Speaker 3

It's long gone, long gone, but literally, but I was always frustated GPS type games, right, games where you're locale mata Right. I did some geocaching and that kind of thing back in the early days, and On this World was an interim game. Was a game in a period

there that was fun. But you mentioned Pokemon Go, because after On This World went away, I started looking for a new game, and I found a game called Ingress, and Ingress is actually the precursor to Pokemon Go because the big piece of data that made Pokemon work where the Ingress location site. So this was a game about again aliens invading sort of thing. But are they beneficial or not? And so you picked a side, are you

working for them or against them? And what they were mainly doing was using GPS to mark up significant locations in communities because those would become nodes in Ingress. Well, when Pokemon Go appeared is from Niantic, the same team, and they basically use the data set from Ingress to create all the locations where.

Speaker 2

The kemon here.

Speaker 1

Wow, so's it's.

Speaker 3

Just suddenly had this valuable GPS data set of curated locations that they get carried to other games that I stopped playing all of those because who's got the time?

Speaker 1

Patrick Hines plays Pokemon Go.

Speaker 3

Pokemon It's one of those quiet be surprised. Lots of people play. It's a quiet little obsession, especially for travelers because there are Pokemon that are specific geographies.

Speaker 1

Well, there are lots of abandoned buildings now because of one of because of COVID, right office space is very cheap right now. You can't give these buildings away in some cases, and certainly shopping malls aren't going anywhere, but they're at least in my neck of the woods, all but abandoned. And you know, everybody's trying to think of something to do with the shopping mall. What do we

do with the shopping mall? And it just kind of occurred to me that maybe some sort of AR future thing I don't think work now, might work with a hollow lens, but you know, I don't know, just sort of occurred to me.

Speaker 3

It's an interesting idea to just have a safe build, a soft walled space, so people crash it and stuff, they're not hurt, and you use AR to make it into whatever game you want to play at the time.

Speaker 1

Yeah, anyway, I think that's about it, Chris. You got anything else that's on your mind that you want to talk about?

Speaker 2

Well, the other thing that has been fun for me as I re engage with you know, the kind of the most fun job I had in my life, but from a you know, from a essentially I've been kind of doing DevRel related work, which you know, has always been my my heart and soul. Anyway, I have been working with Rory Blake. He told me that, yeah, and it has been a ton of fun. I mean, I have reconnected with the Flutter team, and you know, I went to that conference last week and I couldn't walk

five feet with that. Oh how you do it? And how you've been right? That is? That's been fantastic because I have a lot of friends there in the community is wonderful and I just really love Flutter all around, not just the tech, but a nice community. That's right for sure. But then I happened to be hanging out with Rory a little bit and I'm like, hey, you know, you want to do some Flutter work with me, you know?

And he actually when I wanted to have like an existing app that I could drop chat into, because you know, what's the point of helping people build whole new brand news standalone chats? Right? Those exist there, you know, the vendors already have them. But being able to take your existing app and drop in a chat, I think that would be super useful and helpful. But I needed for my talk. I needed an app. I'm like, will you build this for me? And he did? I mean, it

was that was the app I used. He built that initial recipes app for medoschool. He's been doing some he and I have been doing some fun work together and it's just been it's just been really great to reconnect with Rory too.

Speaker 1

And I miss him. I keep in touch with him on texts and stuff. And apparently we were gonna he had another crazy idea to do something, and we almost had a meeting about it, and then it kind of dropped by the wayside. I can't even remember what it was right now, but he's he's just full of good ideas.

Speaker 2

It was he was using llms to build. Actually, Rory has been using generative AI for like five or six years, using kind of pre chat GTP oriented technology, and he's he's found he is an amazing prompt engineer, right. He really best time to make an LLM sing and dance, and so I've learned a ton of really cool tricks from him. And it's all from using this technology prior

where you had to be super specific. And he's built whole command systems and and and he was using that to build up the rules for in the world for a tabletop RPG that he wanted to he wanted to run us through so fantastic.

Speaker 1

Yeah, that's what it was. It was a role playing game.

Speaker 2

Sounds like a show.

Speaker 1

Yeah, we could do a show with him. It be fun to have him back.

Speaker 2

He totally should reach out to.

Speaker 1

I miss him. I missed the blog too. I missed Neapoleon. I understand why he had to shut it down because people weren't taking him seriously as a developer, but man, I miss him.

Speaker 2

It's so funny. Yeah, he's a great I mean, he picked up Fluttering in like a week and he's been doing great stuff. So it's been really great to work with him.

Speaker 3

Yeah, he is the one who coined using objectives. He as Stockholm syndrome.

Speaker 2

And he still loves it. He's still Yeah, he's okay with Stockholm syndrome apparently. Yeah, for sure.

Speaker 1

All right, Chris, it's been a gas and we'll we'll have to keep in touch every so often.

Speaker 2

For sure. Happy to all right you guys. Later, Well, we'll have to keep doing it. We'll have to. I mean, we have to do this, right, I mean, eventually you'll catch up with the present, right, You'll have Show twenty twenty four, and then you'll be podcasting from the future.

Speaker 1

That's where those are the we'll be making predictions.

Speaker 2

Yeah, that's right, that's where I want to make sure I'm on those shows. All right. Cool?

Speaker 1

Well, thanks again Chris, and for you, dear listener, We'll see you next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online it e w op dot com.

Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two thousand and two. And make sure you check out our sponsors. They keep us in business. Now go write some code, See you next time. You got jud Middle Vans doc

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android