AI agents are getting pretty impressive. You might not even realize you're listening to one right now. We work 24 seven to resolve customer inquiries. No hold music, no canned answers, no frustration. Visit Sierra.ai to learn more. Support for the show comes from Alex Partners. The market is evolving at a breakneck speed.
and we're only starting to understand how disruptive forces such as AI, cyber threats, and tariffs will change the game. The winners will be those who prioritize execution and know when to adapt. And for unparalleled insights, they can turn to the Alex Partners Disruption Index. Stay tuned to hear more about it later in the show. In the face of disruption, businesses trust Alex Partners to get things done when it really matters.
Read more on the latest trends and C-suite insights at disruption.alexpartners.com. What do walking 10,000 steps every day, eating five servings of fruits and veggies, and getting eight hours of sleep have in common? They're all healthy choices. But do all healthier choices really pay off?
With prescription plans from CVS Caremark, they do. Their plan designs give your members more choice, which gives your members more ways to get on, stay on and manage their meds. And that helps your business control your costs. Because healthier members are better for business. Go to cmk.co slash access to learn more about helping your members stay adherent. That's cmk.co slash ACCESS.
Hello and welcome to Decoder. This is Alex Heath. For my final episode as your Thursday episode guest host, I recently sat down with Brett Taylor. the CEO of AI startup Sierra and the chairman of OpenAI for a live event in San Francisco hosted by Alex Partners. Very few people have seen the tech industry up close like Brett Taylor has.
He was an early engineer at Google before starting FriendFeed, a social network he sold to Facebook in 2009, where he then served as CTO. He then founded Quip, which he sold to Salesforce. After eventually becoming co-CEO of Salesforce, he left to start Sierra, which is rethinking how businesses use AI for customer support.
Along the way, he led Twitter's board during Elon Musk's takeover and became chairman of the OpenAI board after the firing and rehiring of Sam Altman. As you'll hear in our conversation, Brett is all in on AI. Just this week, Sierra raised a new round of funding, valuing it at $10 billion. In this episode, we get into the origins of Sierra and what it's doing now with AI agents.
I also asked Brett about OpenAI and the overall trajectory of the AI industry. We covered a lot of ground, and I hope you find Brett's perspective as fascinating as I did. Okay, here's my conversation with Brett Taylor. Brett, thanks for being on the show. Thank you for having me. I would like to start by going back to early 2023. You're leaving Salesforce. You were the co-CEO. Talk about that process of deciding.
to make a new company what you looked at why did you land on sierra at that time i happened to announce i was leaving salesforce within a few days of chat gpt coming out so i don't know if there's like you believe in like cosmic forces but i was like every single human being particularly a geek like me when you first use a product like that
It's all you can think about. So I was honestly not 100% sure what I wanted to do when I left. I was trying to leave and then figure it out, which is a good thing to do in life. I immediately just became obsessed with the technology. I was using it personally. Reid Hoffman is a friend of mine and was showing me early versions of GPT-4.
I just couldn't believe the level of empathy and it just truly sounded human. And I had been following AI for years, but I honestly, you know, If you'd told me in October or the month before to define what a large language model is, I would have given you a blank stare. Ended up really realizing that this technology, which I had not been following as closely as I wish I had, I really came to believe was going to change the world. And so...
I knew I wanted to work in it. I didn't know what I wanted to do. But that was okay. It reminded me a little bit of when I first discovered the internet. You know, I think you know it's going to change everything. At least I felt that way. uh i was excited to work in that space and that's all i knew so i ended up having uh lunch with clay bevore who i'd known for 20 years and not
Planning to start a company with him, but found out through the course of lunch that he was equally obsessed. He was working for Sundar at Google at the time. And by the end of lunch, we had a couple more courses than we had originally planned.
uh we decided to start a company and we had no idea what we're going to do but i think it was really based on the premise that when you have a seismic shift in technology a lot of business opportunities present themselves because it kind of shuffles the deck of
both what consumers want, what companies need, and what software vendors are adequate to support that need. If you look at the... advent of the internet it gave birth to some of the largest names in the stock market today like amazon and google it disrupted companies like microsoft who got through it quite strong it disrupted companies like siebel systems a little bit less strong right and so you end up where the
incumbent insurgent dynamic changes quite a bit and huge markets open up. In the case of the internet, it was search and e-commerce. And I think with large language models, and we'll probably talk about it more, so I won't have too long-winded of an answer here.
It really stands to reason that a lot of different markets from software engineering to customer service are going to be transformed, completely upended. What an interesting time to start a company. So we left and gave ourselves a... a few months of just recovering from our jobs and then talked to a lot of customers and decided to build CIRA and at CIRA we're building AI agents for customer experiences so everyone from ADT home security to RAMP
New York to SiriusXM or using it to answer the phone when you call them up or in their digital properties, have a conversation and doing everything from helping you upgrade or downgrade your SiriusXM plan to calling you when your ADT alarm goes off. which I think is pretty exciting. Yeah, talk to me about Sierra and how you work practically with a new customer.
Walk me through that process because this is all a very new field. I mean, customer support is not new, but the way you're doing it is new. So what is unique about how you work with a customer versus how you would have done it at another company before?
I'll start with our business model because I think it helps answer your question, which is one of the things that we do differently than traditional software companies at Syrah is we charge only for outcomes. So for most of our customers, that means when the AI... agent autonomously resolves the case that you called about or chatted in about.
There's a fee for that. And if the agent has to transfer to a real person, it's free. We really like this as a business model. And I think it will become the standard business model for agents because. The word agent comes from agency, and the principle of it is some degree of autonomy.
I think most of the most sophisticated agents will actually start and complete a task, whether it's generating a new lead for your sales team or solving a customer service inquiry or doing a legal analysis for an antitrust review. whatever it might be. And if an AI agent isn't just helping a person be more productive, but actually accomplishing a task.
why not just pay for a job well done? And if you look at most of your companies, if you have a job where the outcome is measurable, like sales, you tend to pay a commission, right? There's not just a salary. And so I think agents sort of being paid on commission, if you will. is actually not only a great incentive alignment between a vendor and a partner and a company, but also just feels from first principles right.
And that's why I think just like the advent of cloud-based software when Mark and Parker started Salesforce and it was a subscription-based service rather than a perpetual license changed the landscape of software, I think the same will happen with agents.
Now going back to how do we work with customers, it kind of... begs the question of what your relationship is between a software vendor and a company if you only get paid when it works there's a certain degree of like arm's length relationship that most software vendors have with their customers and if you've
ever seen someone who's done a big ERP implementation. I don't know much about ERP systems, but apparently it's really hard to implement because everyone I've ever met who's done it, it's taken like two years longer than you expect and a lot more money than you expect. And if you go and... talk to the 10,000 people involved in one of those projects.
systems integrators pointing to the software vendor the software vendors pointing the systems integrator no one's really pointing at the company because they're the one paying the bills so everyone's like oh you're great no everything i'm sure everything's fine And, you know, it's like, you know, success has a thousand fathers, failure is an orphan. But part of the issue is the only...
party in that relationship that cares about the outcome is the company. And so everyone's blaming everyone else. And it requires a good CIO or CTO to navigate that. But you can see all the perverse incentives involved. Maybe the partner's getting paid by the hour. Not a great incentive. The software vendor's already made the sale. So good luck to you getting it successfully deployed. So I think... going towards outcomes based pricing.
demands a different relationship between a software company and the companies that they work with. I heard you just talk about forward deployed, Pius, and I'm not sure that is the only answer. I think it's trendy right now in AI. in part for this reason. No software company wants to be a professional services firm. So you can't turn that knob all the way up to 11 and build a company that I think you want to build. But there is a different level of accountability.
you know with our relationship with our customers we've really focused on sort of a couple different things one is uh product ease of use i think to make your outcome you need to make it as easy as possible to achieve that outcome so we somewhat uniquely in the market have a product for technology teams as well as a product for operations team so you can agents without any technical knowledge at all. And again, trying to empower as many customer experience professionals as possible.
And then on the partnership side, we have a lot of support with what we call agent development. So if you need help getting your agent out the door, we show up in a bus to help you do it. And that's really unique. I'm not sure how everything will play out, but I'm really bought into this vision. And when I talk to our customers, I love the idea that...
you know, they know exactly the value that we're providing for them because they only pay us when the agent works. And I just love the simplicity of that relationship. And I'm really bought into that. So you have hundreds of customers and 50% have revenue over a billion and 20% over 10 billion a year. Is that right? That's correct. Why focus on big customers like that instead of, you know, a huge...
Shopify-like approach to this? Why are you going towards the big companies? Big companies have big problems. I love first principles thinking. And if you are a large consumer brand and you have 100 million... consumers around the globe. Before large language models you literally could not have a conversation with all of them.
If you just do the math, there's a term in call centers called cost per contact, and it's essentially measuring how much all-in labor costs and technology costs to answer the phone or answer the chat. It really depends on how complex the conversation is, what the qualifications of the person answering the phone. It depends whether it's onshore or offshore. But say it's somewhere between $10 and $20 to answer the phone.
For most consumer brands, their average revenue per user is less than that phone call. And so you literally can't afford to have a conversation. It's why if you've ever tried to call any consumer brand, you can't. There's entire websites devoted to finding the phone numbers for many consumer brands. And it's not because they don't care about you. It's just not economical. If everyone who wanted to call them called them, they would go out of business, which is probably not good for you either.
Now, with large language models, that's totally different. You bring down the cost of a phone call by not one, but two orders of magnitude. And all of a sudden, the economics of having a conversation changed dramatically. And so... The reason why we've pursued larger enterprise brands is that's the type of, I would say, step change function and customer experience that is relevant to a company that has tens of millions or hundreds of millions of customers.
And those are the larger companies in the world. And what's really exciting about it is I think a lot of people think when they think about AI agents for customer experience, they think contact center automation. And that's a huge part of it. But if you think about it through the lens of what I just said, you now can have an order or two orders of magnitude more conversations with your customers than you could before for the same cost.
And that's really remarkable. And if you think about all the companies who are competing for, whether it's, let's say, a mobile phone company, you're competing for sort of a fixed pie of customers trying to decide which company you're going to align yourself with. And if you can improve customer retention by 100 basis points, that's a lot. value. If you can reduce your attrition and churn by 500 basis points, that changes the lifetime value equation of your company.
So I think people are thinking about it, I think really the first order effect of reducing the cost of, say, a phone call, which is great. You can save that money, return it to shareholders. But I think the more sophisticated companies are saying, can I actually gain market share? And that's really, really exciting. And that's what we're trying to do for some of the largest brands in the world. We need to take a quick break. We'll be right back. Support for this show comes from .tech domains.
naming your business can feel like naming a child of course one of the main differences is that there could be a lot of different johns in the world but if someone already has the web domain you want you're out of luck with a dot tech domain you can secure the startup name you really want
no compromises. Even better, using .tech signals to customers and investors that your business is tech-focused right from your domain. Just look at some of the companies already on .tech. CES.tech, the world's largest consumer tech event. And 1x.tech, an open AI-backed startup, are just a few of the hundreds of thousands of tech companies worldwide. And it's not just about having a name. It's about standing out. A .tech domain shows that your company is innovative,
forward thinking, and part of the global tech community. Whether you're launching a startup, a new product, or an online service, having the right domain is the first step to making your mark. Support for the show. comes from alex partners disruption is the new economic driver the days of predictable business cycles are over for over 40 years alex partners has helped companies develop winning business strategies amidst uncertainty
One of today's greatest challenges, the rise of AI. As AI reshapes the tech landscape, Alex Partners is committed to helping your company thrive. In their sixth annual Alex Partners Disruption Index, a global survey of 3,200 senior executives, 65% of executives believe AI and machine learning provide positive opportunities for their companies.
And 62% of CEOs expect significant business model changes in the next year. In the face of disruption, businesses trust Alex Partners to get straight to the point and deliver results when it really matters. Read more on the latest trends and C-suite insights at disruption.alexpartners.com.
Support for this show comes from LinkedIn. When you're a small business owner, your business is on your mind 24-7. So when you're hiring, you need a partner that works just as hard as you do. That hiring partner is LinkedIn Jobs. When you clock out, LinkedIn clocks in. LinkedIn makes it easy to post your job for free, share it with your network, and get qualified candidates that you can manage all in one place.
LinkedIn's new features can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights. Either post your job for free or pay to promote. Promoted jobs get three times more qualified applicants. at the end of the day the most important thing to your small business is the quality of candidates and with linkedin you can feel confident that you're getting the best based on linkedin data 72 of smbs using linkedin
Find out why more than 2.5 million small businesses use LinkedIn for hiring today. Find your next great hire on LinkedIn. Post your job for free at linkedin.com slash partner. That's LinkedIn.com slash partner to post your job for free. Terms and conditions apply. We're back. Do you have agents right now that are doing things for customers without human involvement? I'm talking like beyond a chatbot, but actually doing things that have...
economics tied to it or that would be something that you would think a human would be involved in, but it's actually not. Is there an example of this right now? I'll give a few. We have retailers where you can take a photo of your damaged goods. and immediately adjudicate a warranty claim. And it will connect to the inventory system and ship you a new product. You can refinance your home with an AI agent powered in our platform, end to end.
Without a human in the loop. Without a human in the loop. These agents are remarkable at what they're doing. And not only can you take action with an agent built on the Sierra platform, literally 100% of our customers are doing it. To some degree, there's this technique in AI called
retrieval augmented generation which is a fancy way of saying answering questions it turns out that that's kind of a commodity at this point slappy and chat gpt together with a knowledge base is not that hard you know most engineers nowadays could literally do that in a weekend which by the way is mind-blowing science fiction three years ago now it's a weekend project that's welcome to technology it's mind-blowing
Actually being able to put sophisticated guardrails around a regulated process. We work in health insurance payer industry. We work in the provider space. We work with banks. We work with property. casualty insurance companies. And if you're talking about, you know, sophisticated regulated conversations like claims processing, that's not retrieval augmented generation, right? That's a very complex conversation with regulatory oversight.
How do you put AI-based guardrails around it? How do you put deterministic-based guardrails around it? How do you solve the mundane problems of transcription accuracy in 40-plus languages? And it turns out transcription accuracy... It doesn't really matter if it misses the word and or, but it really matters if it's your account number. And so how do you get the hard parts? We do roadside assistance. And it turns out, if you've ever chatted with an AI agent and a car horn honks,
it will often stop talking because it thinks it's being interrupted because it can't distinguish the difference between a car horn and you talking. And, you know, our platform is really designed to solve those problems. Effective guardrails. multilingual conversations over chat and voice deterministic guardrails uh ai based guardrails which is called supervisor models which is really really effective and interesting and
Simple stuff like knowing, hey, that's the television on the background. No one's talking right now. Or that's a car horn. Someone's not interrupting me. And it turns out that, you know, I'm sure in three or four years, that'll be easy. Right now, it's really hard. And so, which is why we have a lot of demand.
for our product. I'm glad you brought up voice. I'd be curious to hear how voice is entering this mix beyond chat. And do you think voice will actually be a bigger piece of the pie for agents than text? I do. Voice is already a bigger part of our platform than text, which is kind of remarkable since we launched it in November of last year. I think it stems from a couple of reasons. One is...
First, I'll just go to human parts of it. I mean, if you watch movies about computers in the future, our vision, science fiction authors' vision of the future, you're usually talking to a computer. I think it is the most...
ergonomic interface. We're all born with it. We all know how to talk. And as a consequence, I think it is quite low friction. It's quite accessible. We talk a lot about the digital divide. And I think if most of the ways you interact with digital technology is just speaking, what a...
great way to just make it accessible to everyone, especially if it's multilingual and patient. And then if you look at the telecommunications industry, the health insurance industry, things like that, a lot of it still goes over the phone. And it's not just... as patients or consumers but you know payers to providers providers to payers excuse me you know a lot of this is still running over the phone and
What AI has done is it's taken one of the oldest analog channels, which is the publicly switched telephone network, and made it digital for the first time. It used to be that almost every company I talked to has a... digital self-service team which is a fancy way of saying hey can you please use our website rather than calling us because gosh it'd be better for you it's better for us it's cheaper it's faster and like there's entire teams devoted to that now it's like
maybe call us. It's all good. It's the same. It turns out the same agent on our website is just picking up the phone, which is kind of crazy. So you've essentially, you know, you always talk about like, you know, TCP IP which is like TCP running over the internet protocol like there's some name for this we've basically put the internet on the phone like that we've just made the phone a channel for digital technology for the first time and so as a consequence you know if you look
Look at, there's a proverb in entrepreneurialism, like you want to make a painkiller, not a vitamin, because people buy painkillers and people think about buying vitamins. This is truly a painkiller. Like, you know, you've just taken the most expensive, the most tedious channel. And everyone hates it too, by the way. It's not...
Even if you talk to the best customer service agent of all time on the phone, it's usually after you've been waiting on hold for 10 minutes. And because the economics of not making a call center where you don't have to wait on hold are just untenable.
So it's just one of those things where consumers, companies, there's no one defending the current landscape of phone calls at all. Everyone hates it on all sides. And now you have this technology that just solves the problem. And so that's why I think it's going to have a big impact. But looking forward, it's really unclear. I don't think most people, and I'm kind of in the center of a lot of this AI stuff. I couldn't tell you where the world is going.
I think it is really exciting. If you look at the way WhatsApp is used in Brazil and India, you wonder with conversational agents, you know, will that style of digital interaction be as pervasive in other markets? I was blown away when I went to Brazil, I don't know, four years ago or something and saw someone do a mortgage over WhatsApp. And I was like, tell me what you're doing. And it's like uploading to PDF.
All of a sudden, if you think about every company in the world's customer experience having a conversational agent... Yeah, maybe every company in the world will have a WhatsApp presence to do that. Or, you know, maybe smart speakers will make a comeback. I think about driving into work and CarPlay and...
CarPlay, I love the product in a lot of ways, but you can't really do anything with it. Imagine triaging your email, having a conversation with a personal agent while you're driving into work, and all of a sudden, your commute just... got super productive. Like it'd be talking to like the personal assistant with a PhD and everything. I mean, that's pretty cool.
So I think what's exciting, we talk about phone because I do think it is the area that is just economically impactful right now. We're making computers conversational. And I think it is a user interface paradigm as much as a technical change. And I've never felt so old. The other day, I was talking to the phone.
uh, like this for people online, I'm touching the phone to the side of my face, which until this moment I thought was normal. My kid was like, you're touching your phone to your face. It would be like someone licking their phone or something. And like all kids.
just talk on the phone differently. And I, I never thought of that as abnormal until that moment. And then I was like, I'm fucking old. Um, and excuse me, I don't know. And You realize that I just think that kids who grew up with these technologies, who never see a rotary dial, they just have a different style of interaction with these new technologies.
younger kids today are going to grow up where, yeah, of course, computers can understand what I say when I talk to them with nuance and sarcasm. And of course, I have a personal AI agent that can go, you know, do my research for me for my next trip. And of course, you know, and... I think we are not even contemplating the second and third order effects that led to my child thinking touching a phone to the side of my face was weird, which still boggles my mind.
But I think we're at the start of a really significant trend. And I'm hopeful in a lot of ways because... I, like many others, read things like The Anxious Generation, and I catch myself being mildly addicted to staring at the glowing screen in my pocket. And you wonder if you fast forward four or five years, will software kind of melt away into the background?
A lot of things that are tedious, like waiting on hold and you can't find a phone number. Would this technology make all that go away? Yeah. Call us anytime. Oh, and it knows everything about me and whether I want to talk to it over chat because I'm on the BART train and I don't want people to hear me or, you know, I want to talk on the phone because I'm holding things in my hands. Like all of that will just.
be available. So I'm excited for it because I think like all technology trends were in that the bottom rung of Maslow's hierarchy of needs. And it's very hard to see. self-realization or whatever the top thing is. But I think we're going to get there relatively quickly. And our hope at Sierra is we can help every company in the world navigate that. Step one is make an amazing customer service experience for your customers that make them feel respected and valued and truly personalized.
Step two is set your company up for whatever the future holds. What does conversational commerce mean? What does it mean when people are doing their consumer research on open AI rather than search engines?
many of you have done that. When you get like a lab result, I just upload into chat GPT immediately before I talk to my doctor, which I don't know how he feels about that, but I promise you a hundred percent of his other patients are doing too. Like the whole world's changing. And so like a lot of what we think about a series, like how do we. We set up every company in the world to be successful in that new world.
Technically, though, are you developing your own models? What is the actual tech secret sauce here that you have? Is it models or what is it? We do a lot of fine-tuning. We don't pre-train any models. I think most AI apply to AI. companies shouldn't. It's a very fast appreciating asset and probably would not produce a meaningful return for your shareholders. But it is quite complex for any given message to one of the agents. on our platform.
Probably 20 plus inference calls just to generate one response, just to give you a sense of the complexity. Lots of different models under the hood. There's not one provider or even one parameter count, which is a measure of sort of the complexity of these models. I think that's the way for an applied AI company like CIRA where the world is going because it's almost like saying what's the right way to store my data and for the technologists in the room there's like a trillion different
databases and data storage systems from Snowflake and Databricks to traditional transactional databases. And we've gotten to the point now where a modern technologist would know, hey, for this use case, this is the correct choice. That's where we're going with, I think, in the applied AI space, not artificial general intelligence, but the applied AI space is
these models are truly pieces of infrastructure. And, you know, sometimes you want something that is really fast and sometimes you want something that's really cheap and sometimes you want something that's really high quality and this price performance latency choice.
there's really an option everywhere in that matrix for whatever you want for your business. And I think it will end up like the database market. It will be practitioners of building these agents and other things. They're not going to be the researchers who know how to pre-train a model. And my intuition for what it's worth is even fine-tuning will wane over time just as the context windows and quality of rules adherence improves in these models.
But there will be what it means to build an application on these models will be like, hey, do you know how to use a database? Not do you know how to write a database? And those are two very different skill sets today. And I think that's kind of where the applied AI market is going. I think we saw with the release of GPT-5 that the models, they're still getting better, but the step changes are not as dramatic as they used to be. Maybe that will change. The space moves.
I don't totally agree with you on this one, but finish your question because I'm rudely interrupting. You're on the board of OpenAI, nor should you. But I guess what I'm saying is... Do you agree with the thesis that the models themselves are kind of commodifying? I mean, you talked about it as infrastructure, but I guess what I'm getting at is what are the second order effects if that is true? If the models are really just becoming...
like anything plug and play. Yes, they have certain attributes that are better, but they're not getting dramatic. They're not dramatically step function changing like they used to. Well, the reason I was disagreeing wasn't being an open AI homer, which I am, by the way. So happy to play that role. Actually, more just saying, I think it really depends on the task. If anyone was using GPT-40 or 401 for coding...
and then swapped in GPT-5 for coding afterwards, you saw a dramatic improvement in performance. So through the lens of that task, it was... very much a step change in performance. And so for people who are using this for coding agents, I think through the lens of that use case, what you said was definitely not true. Absolutely a step change in performance. I planned one of our vacations on ChatGPT earlier this year, and I think I was using 4.0 to do it.
In my guess, if I used GPT-5 for that same trip planning, it would have been like, yeah, okay. It's like, whatever, you know, slightly better. I had a great vacation. So maybe I just didn't have high enough standards. Maybe it would have gotten like a lot better.
I think that we're getting to the point for a lot of tasks. We've reached sufficient intelligence for those tasks. So when new models come out, if you're measuring it relative for... planning my vacation you'd be like gosh i don't see a huge change in the quality of this model if you're trying to discover a new therapy and you're doing drug discovery or you're trying to autonomously write a complex
piece of software, or you're trying to do a complex, asynchronous, agentic task, your perspective on how big of a step change may change. My intuition, but it's just one person's intuition, is that perception of how big of a step change these models are will increasingly be a function of how complex of a problem you're trying to solve with them. And if you think about what it means to build artificial general intelligence, we need some more improvements, right?
There was a really interesting thread on X from an OpenAI researcher who gave it a math paper, and it actually had a relatively novel approach for a type of math I don't understand, so that's the limit of what I can say about that. But it was really interesting. It was really creative. It really had that sort of almost like AlphaGo moment of like, wow, that's interesting. It's sort of novel, new mathematics. Certainly, we want to get to the point of...
developing new AI research, finding new drug therapies, proving some of the unproven math problems in the world. We have some work to do. We haven't gotten to that point. But my guess is for what motivated your comment, probably for a big bunch of tasks, the models have sort of gotten to the point of sufficiency. And I think, so going back to your question, which is, so what does it mean?
I think OpenAI is a mission-driven company. Our mission is to ensure that artificial general intelligence benefits humanity. And we want to work towards beneficial AGI. And we're not there yet. We need to continue to do that research. development. There are parts of it that already are super intelligent, but there are a lot that aren't, and that's really what we're working on. Does it mean for different tasks that Sierra solves or you do in your personal life, we need those really powerful
models maybe not and I think that will just result in a ecosystem of models and and what they're used for but what's exciting just you know around here in san francisco like we're not done yet you know we want to create agi and that's that's really exciting and i think despite the
perception of these models slowing down i'm not sure i don't really subscribe to it and you can see in some of the true research breakthroughs the math olympiad results i mean these are really meaningful new changes that weren't possible with previous models and i think they're are dang exciting we need to take another quick break we'll be right back Support for this show is brought to you by Grammarly.
Getting work done can feel impossible when you're pulled in a million directions. The phone rings, emails pile up, and before you know it, you haven't even touched the one task you actually need to finish. Grammarly wants to help. assistant that boost productivity, which means you can do what you need to get done faster, no matter what or where you're writing. With Grammarly, you can instantly generate well-written drafts with AI to get your ideas down on paper.
Their tools work alongside you and can help you strike the right tone by suggesting a better word or even personalizing your writing based on audience and context in a way that sounds like you, not AI. Grammarly says they're setting the standard for responsible AI you can trust. 90% of professionals say Grammarly has saved them time writing and editing their work. Let Grammarly take the busy work off your plate so you can focus on high-impact work.
Download Grammarly for free at grammarly.com slash podcast. That's grammarly.com slash podcast. Adobe Acrobat Studio. So brand new. Show me all the things PDFs can do. Do your work with ease and speed. PDF spaces is all you need. Do hours of research in an instant. With key insights from an AI assistant. Pick a template with a click. Now you're present. It's time to do your best work with the all new.
Adobe Acrobat Studio. Avoiding your unfinished home projects because you're not sure where to start? Thumbtack knows homes, so you don't have to. Don't know the difference between matte paint finish and satin? Or what that clunking sound from your dryer is? With Thumbtack, you don't have to be a home pro. You just have to hire one. You can hire top-rated pros, see price estimates, and read reviews all on the app. Download today.
And we're back. I'm glad you brought up AGI. I am increasingly of the opinion that no one knows what AGI means. chairman of open ai it actually really matters what you think open ai means and i would love to know uh what agi means and i would love to know what you think it means and if that has changed at all for you like in especially in the last year or so do you have a sense of like
This is AGI. When we have achieved this, we have hit AGI. First, I'll answer the last question, which has it changed. Yes, it has changed for me. I think we are already at what I would have defined AGI at three years ago. Actually, by the way, I think we're at what almost anyone in the world would have defined AGI as three years ago already. There's this thing called the Turing test, which is...
I think, actually, I don't know what the original one was in the paper, but the way it was taught to me in computer science was having a conversation with an AI and having it being basically indistinguishable from human. And we've been...
passed that for years. And we passed this. This was like the big thing in AI for long. We passed that and just like, yeah, forget the Turing test. That was a dumb idea made by the smartest computer scientists of all time. And so we just keep on moving the goalposts because we have... exceeded our own collective expectations about what this technology can do so many times that what we had intuitively thought of as AGI, like we've lapped it like four or five times.
So the way I think about it now, and I may change again, is that in the domain of digital technology and ideas, you know, is it... Are these models exceeding human intelligence or at human intelligence? in almost all domains. I say that in the digital domain of ideas because I think it's one thing to invent new types of math, which I think a lot of people would put in the domain of AGI and superintelligence. you
Interacting with the physical world is a whole different thing, and I think that's a separate problem that's unrelated to intelligence per se. Just being concrete about it, you can invent a new therapy, but a clinical trial is a completely independent process. So I think the intelligence part of it is really what I was trying to find a measure of. And the other part of the G in AGI is generalization.
I don't believe, but, you know, talk to a lot of researchers. That's what's interesting about AI is like some of the smartest people don't agree on all these things. You know, if you make something that is really, really good at math, how good it will be at a lot of other things. And you'll talk to some researchers who think, well, math is sort of the basis of reasoning and it will be great. A lot of things you talk to other people like.
will it generalize to different parts of biology and other things like that? I am more on the camp of, I think, as long as the model isn't trained for something like the math Olympiad and as a byproduct of the model, it will generalize. But I think we'll have to see. And I think we'll see how it generalizes to other forms of science.
things like that. But more and more, just looking at the rate of progress and seeing some of the results coming out of OpenAI and the other research labs, I'm more optimistic. And I'm looking forward to the first true scientific breakthrough coming from an AGI. And I think it will happen in the next couple of years. I don't know. But it feels like it will. It's a better question for the researchers at OpenAI than me. But certainly, if you
start to see some of these early results, it certainly feels possible. Why are people like your old boss, Mark Zuckerberg, now talking about superintelligence? What is the difference here? I mean, this is like a thing when you're out here in Silicon Valley and SF, like now people are saying super intelligence. And it's like, is it because we, like you said, everyone is kind of like, well, we did it, shrug, like Turing test. Rebrand, yeah.
Yeah, rebrand. What is the difference? I don't really understand, to be honest. Superintelligence, I think, literally just means that it is more intelligent than humans. And so, you know... I guess if there's a subtle distinction is if you made something that was generally intelligent and functioned as well as you and me, is that like no lackluster? No offense, Alex, by the way, I think it'd be great if we made it. You're sufficiently intelligent for me.
And so I think it's a higher bar, you know, which is, you know, truly exceptional. There's a few reasons from a research and safety standpoint. It's useful to talk about superintelligence because it's a reminder that. if the models exceed your own capacity to reason,
How do you monitor it? How do you make it safe? And, you know, you really have to use technology to monitor the technology if it exceeds your own capacity to do so. There's lots of precedent in non-AI things. You have lots of things in an airplane or a car monitoring.
for things you can't understand or are operating too fast. But that is a really important area of research. So I think it's useful to talk about. There's sort of the public relations part of it that I don't really have an opinion on. care to think about. But it is useful when you think about safety. And, you know,
there's a real question of like, how do you know that it's aligned if you can't understand it? And how important is it that a human being understand it versus a supervisor AI that we made to understand it? And there's a lot of both technical... And like philosophical questions and all of that, I think are really important to answer as we develop. Yeah. I was at a recent dinner with Sam, Sam Altman. This dinner got a lot of headlines because Sam said.
that he thinks we're in an AI bubble. And his exact quote was someone is going to lose a phenomenal amount of money. We don't know who, and a lot of people are going to make a phenomenal amount of money. um i'm wondering it's like the old marketing quote yeah it's been 50 only 50 of my marketing is useful i just don't know which 50 right um do you agree with that and if so why
Oh, absolutely, yes. I've given this analogy before, so I apologize, Alex, if you've heard it, but I think there's a lot of parallels to the internet bubble. If you look at the internet bubble, a lot of people think about... the flops like pets.com and web van.
Through the lens of the past 30 years, though, we've now gotten most of the largest companies in the world, including Amazon and Google, two of the largest companies in the world. But then you look at how much of Microsoft's market cap is from cloud and others. and you start to look and you say, actually, if you look at the
you know, GDP of the world, how much has actually been created or influenced by the existence of the internet. And one could argue that the all the people in like 1999 were kind of right, you know, kind of, it was as impactful on pretty much by every measure.
And even things like Webman, there's now, you know, as the internet became more distributed, there's really healthy businesses like Instacart and DoorDash and others that are sort of built that are now that the smartphone and the scale of the internet has sort of matured.
So even some of the specific ideas were actually not that bad, but maybe a little early. But if you look at the internet, so like if you were an Amazon shareholder from their IPO to now, you're looking pretty good. If you're a Webvan shareholder, you might... feel a little differently and so both exist at the same time and so i think right now you have just i think the ai modern large language models and modern ai um
are absolutely going to have a huge impact on the economy. If you just look at software engineering and customer service by themselves, I mean, they're just, we haven't seen a world with which we've reached a sufficient number of software engineers, and we probably will with coding agents, you know, just because.
We've taken something scarce and we're making it more plentiful. And what is the market for developing software? I don't know. I don't even know how to measure that because every company in the world is now a software company to some degree. So as a consequence, it's, I think...
Just for me, it almost has to be that there's going to be huge winners in this. And because of the amount of economic opportunity, you just end up with a ton of investors and some companies will fail and some will succeed. If you look at the people who built out... fiber in the early days of the internet. A lot of them went bankrupt, but that fiber ended up getting used, you know, just by the next person, the private equity firm or whatever that bought it.
I think it is both true that AI will transform the economy and I think it will, like the internet, create the economic, like huge amounts of economic value in the future. And I think we're also in a bubble and a lot of people lose a lot of money.
Both are absolutely true at the same time. And there's a lot of historical precedent for both of those being true at the same time. Does it worry you at all that the bubble could be in the sector of AI you're in, in the enterprise? I mean, there was that MIT report that everyone's been talking about where a lot of spend is not.
seeing results i know you have a different you know pricing model that's more geared towards success but like i don't know that it seems like the bubble could be all the enterprises that have rushed in and are spending a ton of money on stuff that's not working
What happens when that reverses? I'll decouple whether I worry about it and that study, which I disagree with because I do worry about it, but I don't worry about that study. So I'll decouple the two. So I'll end with the study because it's like more optimistic than me worrying about my existential, existential.
issues around my business. But I'll start with that. Yeah, I mean, it's weird. So there's this like story that goes on around me about rewriting Google Maps. And it's like mostly true and a little embellished, like many great stories are. And it's interesting to me because
That particular thing is people like to tell the story because they're like, oh, wow, one person wrote a lot of software over a weekend. And now if you've used like codecs or cloud code, you're like, yeah, I can just have an AI agent do that a weekend. So it's like. The thing that's so exciting that was actually part of my own personal identity is now...
literally an AI agent, maybe not quite yet. I wrote some pretty good code, but like, you know, probably in a couple of years, like, yeah, an AI agent could totally do that. And so. It's going to go from, wow, that was impressive to, wow, people did that, you know, like over the next couple of years. So there's the business thing, which is what is the software market of the future? I think it's a really good question because.
If you pull the thread and we reach plateaus like, you know, self-driving cars, we're really excited. It took a long time. So, you know, even smart people can be wrong on these things or are too over optimistic. But with.
agents doing software engineering, we're taking the scarcest resource and one of the highest paying jobs, scarcest resource, and we're literally making AI agents that do that. So what will that do? I have a lot of people say, should I study computer science in school? And I have a bunch of opinions.
I think the answer is yes, but honestly, no one really knows, right? And are we going to reach a world where generating software, and generating is not the hardest part of software, as most software people know, but generating software will largely become a commodity. Maybe a lot of people think that. What does that do to the software market? My hypothesis is actually...
It doesn't change a lot. I don't think when you buy, going back to my ERP, I don't know why ERP systems are on my mind this morning. But if you buy an ERP system... You're not buying like the bits and bytes that did it. You're buying the fact that a lot of companies have their ledger on it and that you can close your books every quarter on it and it's reliable. And they have like patched their servers.
that you know that your cloud-based ERP is not going to have a security vulnerability, and they have these compliance certifications and all these other things that aren't particularly exciting, but they're kind of the boring but important part of enterprise software.
If you could write your own ERP system as a major CPG company, is that a good idea? I'm not totally convinced it is. I always like to say software is like a lawn. You have to tend to it. And so if you build it, you bought it. You have to own it. it and maintain it and deal with all the, you know, there's a new accounting standard that comes out and all of a sudden you're, you have to like do that yourself. So
I think it will change the way we write software. Do I think it will completely upend the landscape of the existence of an enterprise software market? I don't totally believe that. Might be wrong. It's really new. We're just in a really new world because we're taking something...
scarce and making it plentiful i i have thought about this movie a lot recently which is uh and i recommend people watch it which is a hidden figures it's a great disney movie about the putting people on the moon but in particularly the women who
did the math calculations to do it. And their job title was calculator. I'm a calculator. I didn't know until that movie and I watched it with my kids that that was a job title. And there's an interesting one of the women in there, they're putting in an IBM computer, which is like the size of a living room. And she, in a sort of a savvy way, learns how to use punch cards so that she doesn't, you know, like basically for job security.
we're all kind of going through these moments right now. Like I'm a calculator basically. And that story of me with Google maps is like a story of a calculator. Right. And so. But I think the second and third order effects are a little fuzzy. I believe the enterprise software market will change from software to agents. But I believe companies want to buy solutions to their problems, not build software. So I believe the market will continue to exist.
I don't know the basis for the data in that study. I think it was problematic because it conflated people building their own software with AI and buying solutions off the shelf. And I think those are two very different. types of AI adoption. We have basically 100% success rate with our customers doing a proof of concept and going live with our platform. I think it's because we're not selling AI, we're selling customer experience and you just like turn it on.
and it works. There's an amazing company called Harvey somewhere here in San Francisco. I actually don't know where their HQ is. Really great company. I've talked to so many law firms who have deployed Harvey for a lot of their legal analysis. They're all successful. But it's not an AI project. Harvey uses AI. That's the only reason the company can exist. But they're doing it because you want a better antitrust review process. And that's what they're buying from it.
I think we're just in the early days of AI, which there isn't a wonderful vendor for every... problem that you have in your business yet so as a consequence you have to wait or you have to build it yourself and so i don't know what the right answer is i'm not you know and when you're trying to build these things yourself it's just a
just a glacial amount of complexity. And what you end up having is a lot of these, I call it AI tourism, you know, like a lot of people doing, you know, performative AI projects, but to actually go to that last mile is quite difficult. I think the solution will be applied to AI companies. I think if you want to build a better customer experience,
Buy CIRA. If you want a legal AI agent, buy Harvey. And I think we need to go through every use case from supply chain optimization to accounting to maybe an auditor for your accounting department. All of those will be AI agents. there's a company in every single one of those domains and i i think that is the correct way to package and purchase ai software and i think that study
is a measure of all the people who are trying to, which is just lack of maturity in the market. There's just not a solution in every space yet. But there's a few VCs in the room. Hopefully, thanks to all of you, in a couple of years, there will be. And I think that will be the new what.
was software as a service, those new agent companies will be that next frontier of business solutions for enterprises. All right, Brad, we have to leave it there. Thank you so much. Thanks for having me. All right.
I'd like to thank Brett Taylor for taking the time to speak with me and thank you for tuning in. I've really enjoyed guest hosting Decoder these last several weeks. I hope you've enjoyed the episodes. If you'd like to let us know what you thought about this show or what else you'd like us to cover, drop us a line.
You can email us at decoder at theverge.com. The team really does read every email. We also have a TikTok and an Instagram. Check those out at decoder pod. If you like decoder, please share it with your friends and subscribe wherever you get your podcasts. Decoder is a production of The Verge and is part of the Vox Media Podcast Network. Our producers are Kate Cox and Nick Stat. Our editor is Ursa Wright. The Decoder music is by Breakmaster Cylinder.
Adobe Acrobat Studio, so brand new. Show me all the things PDFs can do. Do your work with ease and speed. PDF spaces is all you need. Do hours of research in an instant. With key insights from an AI assistant. Take a template with a click. Now you're present. Do that. Do that.
With Acrobat, now you can do that. Do that with the all-new Acrobat. It's time to do your best work with the all-new Adobe Acrobat Studio. Adobe Acrobat Studio, so brand new. Show me all the things PDFs can do. Do your work with ease and speed. pdf spaces is all you need do hours of research in an instant with key insights from an ai assistant pick a template with a click now your prezzo looks super slick close that deal yeah you won do that doing that did that done now you can do
It's time to do your best work with the all-new Adobe Acrobat Studio. Support for the show comes from Mercury. Because to you, it's more than an invoice. It's your hard work becoming revenue. It's more than a wire. It's payroll for your team. It's more than a deposit. It's landing your fundraise. The truth is, banking can do more. Thank you. Thank you. Banking that does more.