Support for this show comes from Wix Studio Designers and Devs You might be able to do your thing better on Wix Studio, a web platform with everything you need to deliver bespoke sites hyper efficiently. Design teams get a ton of smart features that can take the grind out of web creation without it costing per pixel control. Dev teams, you get a zero setup developer first environment combined with an AI code assistant and your preferred IDE
for rapid deployment. Search Wix Studio today to explore the full range of features. At the Coca-Cola Company, Cureg Doctor Pepper and PepsiCo, some of our bottles can be remade in a whole new way, using 100% recycled plastic, new bottles, using no new plastic, except the caps and labels. Learn more at madetoberemay.org. Hello and welcome to Decoder. I'm Neil Aipatel, Editor and Chief of The Verge, and Decoder is my show about big ideas and other problems.
Today I'm talking with Aidan Gomez, the CEO and co-founder of Cohere. Notably Aidan used to work at Google where he was one of the authors of the paper called Attention is All You Need, the described Transformers and really kicked off the LLM revolution in AI. Cohere is one of the busiest AI startups around right now, but its focus is a little different than many of the others. Unlike, say, OpenAI, it's not making consumer products at all.
Aidan co-hearers focused on the enterprise market and making AI products for big companies. Aidan and I talked a lot about that difference, and how it potentially gives Cohere a much clearer path to profitability than some of its competitors. Computing power is expensive, especially in AI, but you'll hear Aidan explain that the way Cohere is structured gives his company an advantage, because it doesn't have to spend quite as much money to build
its models. One interesting thing you'll also hear Aidan talk about is the benefit of competition in the enterprise space. A lot of the tech industry is very highly concentrated, with only a handful of options for various consumer services. Regular Dakota listeners have heard us talk about this a lot before, especially in AI. If you want GPUs to power AI models, you're probably buying something from Nvidia, ideally a big stack of Nvidia
H100s, if you can even get any. But Aidan points out that his enterprise customers are both risk-averse and price-sensitive. They want Cohere to be operating in a competitive landscape, because then they can secure better deals instead of being locked into a single provider. So Cohere has had to be competitive from the beginning, which Aidan says has made the
company thrive. Aidan and I also talked a lot about what AI can and can't do. We agreed that it's definitely not there yet, it's not ready, whatever you think the future might hold. And Aidan says even if you're training AI on a limited, specific, deep set of data, like contract law, you still need a human in the loop. But he sees a time when AI will eventually surpass human knowledge even in fields of medicine. You know, I think about me, you know, I'm
very skeptical of that idea. And then there's the really big tension you'll hear us get into all the way through this episode. Up until recently, computers have been deterministic. If you give computers a certain input, you usually know exactly what output you're going to get.
It's predictable. There's a logic to it. But if we start talking to computers with human language and getting human language back, well, human language is pretty messy, and it makes the entire process of knowing what to put in and what exactly we're going to get out of our computers different than it's ever been before. And I really wanted to know if Aidan thinks that LLMs as we have them today can bear the weight of all of our expectations for AI
even that messiness. Okay. Aidan Gomez, CEO of Cohear. Here we go. Aidan Gomez, you are a co-founder and CEO of Cohear. Welcome to the coder. Thank you. Thank you. Excited to be here. I'm excited to talk to you. It feels like Cohear has a very different approach to AI. You have a very different approach to AI. I want to talk about all of that, the competitive landscape. I'm dying to know if you think it's a bubble. But I want to start
with a very big question. You are one of the eight co-authors on the paper that started this all. Attention is all you need. That's the paper that described Transformers at Google. That's the T and GPs. I always ask this question of people who have been on the journey. Like I think about music documentaries. There's the kids in the garage playing their instruments and then they're in the stadium and no one ever talks about X2. You were in the garage,
right? You're writing this paper, you're developing Transformers. When did you know this technology would be the basis for everything that's come in the modern AI boom? I think it was not clear to me. Certainly while we were doing the work, it felt like a regular research project. You know, we were making good progress on translation, which is what we built the Transformer War. That was a pretty well-understood, well-known problem. We already had Google Translate.
We wanted to make it a little bit better. We improved the accuracy by a few percent by creating this architecture. I thought it was done. That was the contribution. It was that we improved translation a little bit. It happened later that we started to see the community pick up the architecture and start to apply it to way more stuff than we had ever contemplated when building it. So I think it took about a year for the community to take notice. First, it was published. It went
into an academic conference. Then we just started to see this snowball effect, where everyone started adapting it to new use cases. It wasn't just for translation. It started being used for all of these other NLP or natural language processing applications. Then we saw it applied towards language modeling and language representation. That was really the spark where things started to change. One question I have in that process that you're describing. This is a very familiar
abstract process for any new technology product. People develop a new technology for a purpose. A lot of people get their hands on it. The purpose changes the use cases expand beyond what the inventors ever thought of. The next version of the technology gets tailored to what the users are doing. Tell me about that. I want to talk about Cohe here and the actual company you're building and enterprise use cases and all that. That turn with transformers and LLMs and what people think
they can do now. It feels like the gap is actually widening. With the technology can do and what people want it to do because it's inspiring in its way to get a computer to talk to you. It feels like that gap is widening. I'm just wondering since you were there at the beginning and you saw that first turn, how did you feel about that first turn and do you think we're getting beyond what the technology can do? I like the description. The idea that the gap is widening because it's
inspired so many people. I think the expectations are increasing dramatically and it's funny that it works that way. The technology, it's improved massively and it's changed in terms of its utility dramatically. There's no way seven years ago when we created the transformer, any of us thought we'd be here in seven years. It happened much, much faster than then anticipated. But that being said, that just raises the bar in terms of what people expect. It's a language model and language is
the intellectual interface that we use. It's very easy to personify the technology and you expect from the tech, what you expect from a human. I think that's reasonable. It's behaving in ways that are genuinely intelligent. All of us who are working on this technology project of realizing language models and bringing AI into reality, we're all pushing for the same thing and our expectations have raised. I like that characterization that the bar for AI has risen. I think over the past seven
years, there have been so many naysayers of AI. It's not going to continue getting better. The methods that we're using, this architecture that we're using, it's not the right one, etc. etc. They would set bars saying, well, it can't do this. But then fast forward three months, the model can do that. They say, okay, well, it can do that, but it can't do. So that goalpost moving process is just kept going for seven years. We've just kept beating expectations and surpassing
what we thought was possible with the technology. That being said, there's a long way to go. Like you point out, I think there is still laws to the technology. And one of the things that I'm nervous about is because the technology is so similar to what it feels like to interact with a human that people overestimate it or trust it more than they should and put it into deployment scenarios that it's not ready for. That actually brings me to one of my core questions that I think
I'm going to start asking everybody who works in AI. You mentioned intelligence, you mentioned the capabilities. You said they were reasoning, I think. Do you think language is the same as intelligence here? Or do you think they're evolving in the technology on different paths that we're getting better at and more capable of having computers use language? And then intelligence is increasing at a different rate or maybe plateauing? I mean, I don't think that intelligence is the
same thing as language. I think in order to understand language, you need a high degree of intelligence. There's a question as to whether these models understand language or whether they're sort of just parroting it back to us. This is the other very famous paper at Google, right? The stochastic parrots paper. It caused a lot of controversy, right? But the claim in that paper is these things are just repeating words back at us. And there isn't some deeper intelligence
there. And actually, by repeating things back at us, they will express the bias of the things are trained on. That's what intelligence gets you over. That's the thing is you can learn a lot of things and your intelligence will help you transcend the things that you've learned. Again, you were there beating. That's kind of why I want to start at that. Is that how you see it that the models can transcend their training? Or they will always kind of be limited by that?
I would argue that humans do a lot of parroting and they have biases. To a large extent, I think the intelligence systems that we do know exist humans. We do a lot of this. There's that saying we're the average of like the 10 books we read or the 10 people closest to us. And so we model ourselves off of what we've seen in the world. At the same time, humans are or genuinely creative. We do stuff that we've never seen before. We go beyond the training data,
right? And I think that's what people mean when they say intelligence is that you're able to discover new truths. And that's more than just parroting back what you've already seen. I think that these models don't just paraback what they've seen. I think that they're able to extrapolate beyond what we've shown them. Recognize patterns in the data and apply that. Those patterns to new inputs that they've never seen before. I think definitively at this stage, we can say we're past the
stochastic parrot hypothesis. Do you think that's, is that an emergent behavior of these models that has surprised you? Is that something you thought about when you were working on transformers at the beginning? You said it's been a journey over seven years. When did that realization hit for you? There were a few moments. I'm very early on at Google. We started training language models with transformers. So we just started playing around with it. And it wasn't
the same sort of language model that you interact with today. It was just trained on Wikipedia. And so the model could only write Wikipedia articles. But I remember this. It might have been the most useful version of Altness. It was a much simpler version of a language model. And it was a shock to see it. Because at that stage, back then, computers could barely string a sentence together properly. Nothing they wrote made sense. They were spelling mistakes.
It was just like a lot of noise. And then suddenly, one day, we kind of woke up, sampled from the model. And it was writing entire documents as fluently as a human. That just came as this huge shock to me, like a moment of awe with the technology. And that's just repeated again and again. I keep having these moments where, yeah, you are nervous that maybe this thing is just a stochastic parrot. Maybe it'll never be able to reach the utility that
we want it to reach because there's some sort of fundamental bottleneck there. We can't make the things smarter. We can't push it beyond a particular capability. Every time that we improve these models, it breaks through these thresholds. And so at this point, I think that that breakthrough is going to continue. And anything that we want these models to be able to do, given enough time, given enough resources, will be able to deliver that. It's important to remember that,
you know, we're not at that end state already. And so the technology, we shouldn't be letting these models prescribe drugs to people without human oversight. Like, they're very obvious applications where the text is not ready. One day, it might be right. Like, at some point, you know, genuinely, you might have a model that has read all of humanity's knowledge about medicine. And you're actually going to trust it more than you trust a human doctor who's only been
able to, given the limited time that humans have, read a subset. So I view that as a very possible future. But today, the reality that exists, I really hope that no one is taking medical advice from the models and the humans still in the loop. So you have to be conscious of the limitations that exist with what's today. That's very much what I mean when I say that the gap is widening. And I think that brings us to co here. I wanted to start with what I think of his act two. Because
act two traditionally gets so little attention. But the I built a thing and then I turned it into a business and that was hard for seven years. I feel like gets so little attention. But now it's easier to understand what you're trying to do at co here. Co here is very enterprise focus. Can you describe the company? We build models and we make them available for enterprises. So we're not trying to do like a consumer, like a chat GPT competitor. What we're trying to build is a platform
that lets enterprises adopt this technology. And we're really pushing on I think two fronts. The first is, okay, we just got to the state where computers can understand language. They can speak to us now. And so that should mean that pretty much every computational system, every single product that we've built, we can refactor it to have that interface and to allow humans to interact with it through their language. And so we want to help industry adopt this tech, implement language
as an interface into all of their products. That's the first one. It's very external facing for these companies. The second one is internally facing and it's productivity. So this tech, I think it's becoming clear that we're entering into a new industrial revolution that's focused instead of taking physical labor off the backs of humanity. It's taking intellectual labor. These models are smart. They can do complicated work that requires reasoning, deep understanding,
access to a lot of data and information, which is what a lot of humans do today in work. And so we can take that labor and we can put it on these models and make organizations dramatically more productive. So those are the two things that we're trying to accomplish. One of the things about using language to talk to computers and having computers speak to you in language is I would say famously human language is prone to misunderstandings. Most of history's great stories involve some
deep misunderstanding and human language. It's non-deterministic in that way. The way we use language is really fuzzy. Program computers is historically very deterministic. It's very predictable. How do you think philosophically about bridging that gap? We're going to sell you a product that makes the interface to your business a little fuzzier, a little messier, perhaps a little more prone to misunderstanding, but it'll be more comfortable. How do you think about that gap as
you go into market with a product like this? Yeah, the way that you program with this technology, it's non-deterministic. It's stochastic. It's probability is it like there's literally a chance that it could say everything, right? There's some probability that it will say something completely absurd. I think our job as technology builders is to introduce good tools for controllability, so that that probability is one in many, many trillion. You just in practice, never observe it.
That being said, I think that businesses are used to stochastic entities and conducting their business using that because we're humans, right? We're sales people and sales people. The most stochastic of all. Yeah, marketers, we're very used to that. The world is robust to having that present. We're robust to noise and error and mistakes. What we do is humans, when we're interacting with a salesperson, hopefully you can trust every salesperson,
right? Hopefully they never mislead or over claim, but in reality, they do mislead and over claim sometimes. When you're being pitched to by a salesperson, you apply appropriate bounds around what they're saying. I'm not going to completely take whatever you say as the gospel. I think that the world is actually super robust to having systems like these play apart. While it might seem scary at first because it's like, oh, well, computer programs are completely deterministic. I
know exactly what they're going to output when I put in this input. That's actually unusual. That's weird in our world. It's super weird to have truly deterministic systems. It's a new thing and we're actually getting back to something that's much more natural. When I look at a jailbreak prompt for one of these chatbots and you can see this sort of leading prompt. You are a chatbot. Don't say these things. Make sure you answer in this way. Here are some stuff that's completely out of bounds
for you. Those get leaked all the time and I find them fascinating to you. They're all from very long. My first thought every time is this is an absolutely bananas way to program a computer. You're going to talk to it like a somewhat irresponsible teenager and say like, this is your role and hopefully it follows it. Maybe there's a one in a trillion chance that won't follow it and I'll say something crazy, but there's still a one in a trillion chance that even after all of these
instructions to a computer, it'll go completely off the rails. I think the internet community delights in making these chatbots go off the rails. You're selling enterprise software. You're going into big companies and saying, here's our models. We can control them so that reduces that. Reduces the possibility of chaos, but we want you to reinvent your business with these tools because they will make some things better. It will make your productivity higher. It'll make
your customers happier. Are you sensing a gap there? That's the big cultural reset that I think about. Computers are deterministic. We've built modernity around the very deterministic nature of computers. You know what outputs who get versus what inputs. Now you have to say to a bunch of businesses, spend money, risk your business on a new way of thinking about computers. You've described it's a big change. Is that working? Are you seeing excitement around that? Are you seeing
a pushback on it? What's the response? That goes back to what I was saying about knowing where to deploy the technology and what it's ready for, what it's reliable enough for. So there are places where we don't want to put this technology today because it's not robust enough. I'm lucky in that because co-hearers is an enterprise company, we work really closely with our customers. It's
not like we just throw it out there and hope that they succeed. We're very involved in the process in helping them think through where they deploy it and what change they're trying to drive. There's no one who's giving access to their bank account to these models to manage their money. I hope. So there are places where you want to terminus them. You want extremely high confidence guardrails. So you're not going to just put a model there and let it decide what it wants to do.
In the vast majority of use cases and applications, it's actually about augmenting humans. So you have a human employee who is trying to get some work done and they're going to use this thing as a tool to basically make themselves faster, more effective, more efficient, more accurate. And it's augmenting them, but they're still in the loop. So they're still checking that work. They're still making sure
that the model is producing something that's sensible. At the end of the day, they're accountable to the decisions that they make and what they do with that tool as part of their job. I think what you're pointing to is what happens in those applications where a human is completely out of the loop and we're really offloading the full job onto these models. That's a ways away. And I think that you're right. We need to have much more trust and controllability and the ability to set up
those guardrails so that they behave more deterministically. You pointed to the prompting of these models and how it's funny that the way you actually control them is by talking to them. Like, yeah, like you said, like a talking to a team. It's a stern lecture. It's crazy to me every time I look at one. It's somewhat magical. Right. The fact that you can actually control the behavior of these
things effectively using that method. Like, that's pretty magical. But beyond that, beyond just prompting and talking to the same, you can set up controls and guardrails outside of the model. You can have models watching this model and intervening and blocking it from doing certain actions in certain cases. And so I think what we need to start changing is our conception of is this one model. It's like one AI, which we're just handing control over to. What if it messes up? What if everything goes
wrong? In reality, it's going to be much larger systems that include observation systems that are deterministic and check for patterns of failure. If the model does this and this, you know it's gone off the rails. And so shut it down. Right. And that's a completely deterministic check. And then you'll have other models which can observe and sort of get feedback to the model, prevent it from
taking actions. If it's, you know, going astray, the programming paradigm or the technology paradigm, it started off as what you're describing, which is there's a model and you're going to apply it to some use case. So it's just the model in the use case. It's shifting towards bigger systems with much more complexity and components. And it's less like there's an AI that you're applying to go to work for you. And it's actually a sophisticated piece of software that you're deploying to go to
work for you. We have to take a quick break. We'll be back in a minute. Support for decoder comes from Mint Mobile. They say time is money, but do you know what else is money? Money, money is money. And since you're taking the time to listen to this ad, we figured you should get something back like saving a bunch of money on your cell phone bill. Here's one way you can do that. Check out Mint Mobile. Mint Mobile offers a wireless plan for just 15 bucks a
month when you purchase a three month plan. All Mint Mobile plans come with high speed data and unlimited talk and text delivered on the nation's largest 5G network. You can make the switch and keep the phone and number you have right now along with all your existing contacts. You can get this new customer offer and a new three month unlimited wireless plan for just 15 bucks a month by going to MintMobile.com slash decoder. That's MintMobile.com slash decoder. You can cut your wireless
bill to 15 bucks a month at MintMobile.com slash decoder. $45 upfront payment required a equivalent to $15 a month. New customers on first three month plan only. Speed slower above 40 gigabits on unlimited plan, additional taxes, fees and restrictions apply. See Mint Mobile for details. Support for decoder comes from BetterHelp. Look, we're more than halfway through the year, which feels wild. I'm sure you've had plenty of triumphs and plenty of disappointments too,
but right now we really want to focus on the highs. For me personally, I'm really proud of the boundaries I've set this year in both my work and personal lives. If you're looking to find more opportunities to take a moment, celebrate what's going well and adjust the things that need some tweaks. Online therapy from BetterHelp might be worth checking out. BetterHelp Online Therapy offers a safe space to work through stress and set achievable, realistic goals for the rest of the year.
BetterHelp is a convenient, affordable way to give talk therapy a try. It's entirely online and flexible enough to fit into any schedule. You can fill out a brief questionnaire to get matched with the licensed therapist ready to help you lighten your emotional load. Take a moment with BetterHelp. Visit BetterHelp.com slash verge today to get 10% off your first month. That's BetterHelp, help.com slash verge.
Support for this show comes from Slack. You're a growing business and you can't afford to slow down. If anything, you could probably use a few more hours in the day. That's why the most successful growing businesses are working together in Slack. Slack is where work happens, with all your people, data, and information in one AI-powered place. Start a call instantly in Huttles and ditch cumbersome calendar invites, or build an automation with workflow builder to take routine tasks
off your plate. No coding required. Grow your business and Slack. Visit Slack.com to get started. Welcome back. I'm talking to cohere CEO Aiden Gomez about a pretty foundational shift in the way people and computers interact. Up until now, we've been used to what you would call a deterministic model, where you give a computer a certain input and you know what kind of output you'll get. The generative AI is really changing how we interact with computers. Coherent has two
models. Coherent command. Coherent embed. You're obviously working on those models. You're training them, developing them, applying them to customers. How much of the company is spending its time on this other thing you're describing, building the deterministic control systems, figuring out how to chain models together to provide more predictability? I can speak to the enterprise world and their enterprises are super risk-averse. They're always
looking for opportunities, but they're extremely risk-averse. That's the first thing that they're thinking about. Pretty much every initial conversation I have with a customer is about what you're asking. That's the first thing that comes to a person's mind is, can I use the system reliably? We need to show them, okay, well, let's look at the specific use case that you're pursuing. Maybe it is assisting lawyers with contract drafting, which is something that we do with
a company called Borderless. In that case, you need a human in the loop. There's no way that you're going to be sending out contracts that are completely synthetically generated with no oversight. We come in and we try to help guide and educate in terms of the types of systems that you can build for oversight, whether it's humans in the loop or more automated systems to help
de-risk things. That for enterprises at least, with consumers, I think it's a little bit different, but for enterprises, the very first question we'll get from a board or a C-suite at the company, it's going to be related to risk and protecting against it. So, upon that to cohere and how you're developing your products, how is cohere structured? Is that
reflected in how the company is structured? I think so. We have safety teams internally that are focused on making our models more controllable, less biased, and at the same time on the go-to-market side, that project, because this technology is new, it's an education campaign. It's getting people familiar with what this technology is. It's a paradigm shift in terms of how you build
software and technology. We were saying it's stochastic. So, educating people about that, we build stuff like the LLMU, which is like the LLM University, where we teach people, but the pitfalls might be with the tech and how to protect against those. So, for us, I think our structure is focused on helping the market get familiar with the technology and its limitations while they're adopting it. How many people are at cohere? It's always shocking to say, but we're
about 350 at the moment, which is insane to me. It's only insane because you're the founder. Yeah. It was like yesterday. It was just Nick Ivan and I. We were in this tiny little, there was basically a closet. I don't know how many square meters it was, but single digits. We had a company off, say, a few weeks back, and it was hundreds of people. Hundreds of people building this thing alongside you. You do ask yourself, how did we get here? How did all this
happen? It's really fun. Of those 350 people, what's the split? How many are engineering, how many are sales enterprise companies need a lot of post-sales support? What's the split there? Yeah, the overwhelming majority are engineers. Very recently, the go-to-market team has exploded. I think that market is just going into production now with this technology. It's starting to actually hit the hands of employees, of customers, of users. Last year, it was the year of the POC,
or the proof of concept. Everyone became aware of the technology. We've been working on this for nearly five years now, but it was only 2023 when people really, the general public, noticed it, started to use it and fell in love with the technology. That led to enterprises. They're people too. They're hearing about this. They're using this. They're thinking of how they can adopt the technology. They got excited about it. They spun up these tests, these POCs,
to try and build a deeper understanding and familiarity with the tech. Those POCs, like the initial cohort of them, they're complete now. People like the stuff that they've built. Now, it's a project of taking those pre-deployment tests and actually getting them into production in a scalable way. That's the majority of our focus is scalability in production.
Is that scalability in... Okay, we can add five more customers without a massive incremental cost as its scalability in compute, as its scalability in how fast you're designing the solutions for people? Or is it everything? Yes, it's all of the above. I would say one thing is, as a lot of people may have heard, the tech is expensive to build and expensive to run. We're talking like hundreds of billions, trillions of tunable parameters inside just a single one of these models. It requires
a lot of memory to store these things. It requires tons of compute to run them. So, during the PSC phase last year, in a PSC, you have five users. So, scalability doesn't matter. The cost is kind of irrelevant. You just want to build a proof of what is possible. But then, if you like what you've built and you're going to push this thing into production and you go to your finance office, you say, yeah, okay, here's what it costs for five users. We'd like to put it in front of all 10
million. The numbers are they don't compute, right? Not economically viable. And so, for cohere, we've been focused on not making the largest possible model, but instead making the model that market can actually consume and is actually useful for enterprises. And so, that's doing what you say, which is focusing on compression, speed, scalability, and ensuring that we can actually build a technology that market can consume because over the past few years, a lot of this stuff
has been a research project without large-scale deployment. And so, the concerns around scalability hadn't yet emerged. But we knew for enterprises, very cost-sensitive entities, very economically driven, right? If they can't make the numbers work in terms of return on an investment, I'm going to spend this much to deploy this new product feature. If that doesn't work, they don't adopt it. It's very simple. And so, we've been focused on building a category of technology that
is actually right-sized for market. You obviously started all this work at Google. Google has an infinite amount of resources. Google also has massive operational scale, their ability to optimize and bring down the cost curve of new technologies like this is very high, given their infrastructure and their reach. What made you want to go and do this on your own without their scale? For Nick Ivan and I, Nick was also at Google. We were both working for Jeff Hinton and Toronto.
He was the guy who created neural networks. The technology that underpins all of this, that underpins LLM, that underpins pretty much every AI that you interact with on a daily basis. We loved it there, but I think what was missing was a product ambition and a velocity that we felt was necessary for us to execute. And so, we had to start cooking. Google was a great place to do research. And I think it has some of the smartest people in AI on the face of the planet.
But for us, the world needed something new. The world needed cohere and the ability to adopt this technology from an independent, an organization that wasn't tied to any one cloud, any one hyper-scaler. Something that's very important to enterprises is optionality. If you're like a CTO at a large retailer, you're probably spending half a billion dollars, a billion dollars, on one of the cloud providers for your compute. And in order to get a good deal, you need to be able
to plausibly flip between providers. Otherwise, they're just going to squeeze you, you know, add and finum and rip you off. So you need to be able to flip. And so you hate buying proprietary technology that's only available on one stack. You really want to preserve your optionality to flip between. And that's what cohere allows for. So because we're independent, because we haven't gotten locked into one of these big clouds, we're able to offer that to market, which is super
important to them. Let me ask you the D decoder question. We've talked a lot about the journey to get here, the challenges you need to solve. You're a founder. You've got 350 people now. How do you make decisions? What's your framework to make decisions? What's my framework? I flip a coin. You're allowed to say, I asked a lot of, I asked a lot of, I want to really say. I'm lucky in that I'm surrounded by people who are way smarter than me. Like I'm just surrounded
by them. Everyone at cohere is better than me at the thing that they do. And so I have this luxury of just being able to go ask people for advice. Whether it's the board of cohere or the executive team of cohere or the ICs, like the people who are actually doing the real work, I can just ask for advice and their takes. And I can kind of be an aggregation point. And when there are ties, then it just comes down to me. And usually it's just going with my intuition about what's right.
But fortunately, I don't have to make a lot of decisions because I have way smarter people that surround me. There are some big decisions you do have to make. You just, for example, announce two models in April, command R. There's one called rerank three. Models are costly to train, they're costly to develop. You've got to rebuild your technology around the new models and its capabilities. Those are big calls. It feels like every AI company is sort of racing to develop
the next generation of models. How are you thinking about that investment over time? You talked a lot about the cost of a proof of concept versus an operationalized thing. New models are the most expensive of them all. How are you thinking about those costs? Yeah, it's really, really expensive. What could give us a number? Can you give us a number? I don't know if I can give a specific number, but I can say order of magnitude. In order to do what we do, you need to spend hundreds of
millions of dollars a year. That's what it costs. And that's like, for us, we think that we're hyper-capital efficient, extremely capital efficient. We're not trying to build models that are too big for market that are kind of superficial. We're trying to build stuff that market can actually consume. And so because of that, it's cheaper for us and we can focus our capital more. There are folks out there spending many, many billions of dollars a year to build their models.
Yeah, I think that's a huge consideration for us. We're lucky in that we're small, relatively speaking, our strategy lends itself towards more capital efficiency and actually building the technology that market needs as opposed to building perspective research projects. We focus on actual tech that the market can consume. But yeah, like you say, it's hugely expensive. And the way that we solve that is a, raising money, right? Like getting the capital to actually pay
for the work that we need to do. And then b, choosing to focus in our technology. So instead of trying to do everything, instead of trying to nail every single potential application of the technology, we focus on the patterns or use cases that we think are, are going to be dominant or our dominant already in how people use it. One example of that is, rag or AG or retrieval augmented generation. It's this idea that these models are trained on the internet. They have a lot of
knowledge about public facts and that type of thing. But if you're an enterprise, you want it to know about you, you want to know about your enterprise, your proprietary information. And so what rag lets you do is sit your model down next to your private databases or stores of your knowledge and connect the two. So that pattern, that's something that is ubiquitous. Anyone who's adopting this technology, they want it to have access to their internal information and knowledge. And so
we focused on getting extremely, extremely good at that pattern. And we're fortunate we have the guy who invented rag Patrick Lewis leading that effort at co here. But because we're able to carve away a lot of the space of potential applications, it lets us be grammatically more efficient in what we want to do and what we want to build with these models. So that will continue into the future. But that's like I said, it's still a multi-hundred million dollar a year project.
So it's very, very capital intensive. I said I wanted to ask you if this was a bubble. I'll start with co here specifically, but then I want to talk about the industry in general. So it's multi-hundred million dollars a year just to run the company to just run the compute. That's before you've paid a salary and the AI salaries are pretty high. So that's another bunch of money you got to pay. And you got to pay for office space. You got to buy laptops, right? There's a whole bunch of
stuff. There's a lot of stuff. But just the compute is a hundred, hundreds of millions of dollars, a year. That's the run rate on just the compute. Do you see a path to revenue that justifies that amount of just pure run rate and compute? Absolutely. But we wouldn't be building it if we did it. I think your competitors are like, it'll come. There's a lot of wishful thinking. So starting with that question with you, because you started enterprise business, I'm assuming you see it
much clearer path. But in the industry, I see a lot of wishful thinking that it'll just arrive down the road. I completely agree. So what does co here's path specifically? Like I said, we're dramatically more capital efficient. We might spend 20% of our competitors spend on compute. But what we build is very, very good at the stuff that the market actually wants. And so we can chop off 80% of the expense and deliver something that is just as compelling to market. That's a core
piece of our strategy of how we're going to do this. And of course, if we didn't see a business that was many, many billions in revenue, we just wouldn't be building this. What's the path to billions in revenue? What's the timeline? The timeline, I think billions of revenue, I don't know about specifically, or co here, how much I can disclose. It's closer than you think. There's a lot of spend that's being activated in market. Certainly already, there's billions
being spent on this technology in the enterprise market today. And a lot of that goes to the compute as opposed to the models. But there's a lot of spending happening in AI. And like I was saying, last year was very much a POC phase. And POC spend is 5%, 3% of what a production workload looks like. But now those production workloads are coming online. This technology is hitting products that interact with tens, hundreds of millions of people. And so it's really becoming ubiquitous.
It's close. It's in a matter of a few years. And it's typical for like a technology adoption cycle. Enterprises are slow. They tend to be slow to adopt. They're very sticky. Like once they've adopted something, it's there in perpetuity. But it takes them a while to build confidence and actually make the decision to commit and adopt the technology. So it's only been about a year and a half since people woke up to the tech. But in that year and a half, we're now
starting to see real serious adoption, serious production workloads. We have to take another quick break. When we come back, we'll dig into one of the biggest challenges for any enterprise company, managing risk. Remember just a few days ago when for the first time ever, a United States president got convicted of a felony of 34 felonies. The first thing he said was, of course, the whole trial was rigged. But the second thing he said was, he was weirdly talking all of a
sudden about immigration. The next day he spoke about his guilty verdicts again. This time, immigration was the first thing he wanted to talk about. When you look at our country, what's happening where millions and millions of people are flowing in from all parts of the world, not yourself, America, from Africa, from Asia, from the Middle East. It's an election year and the leading Republican candidate, maybe the leading candidate period wants to make it all about
immigration. So we're going to do the same on today's, explained with two episodes this week. Come over and join us. Give them a listen. Welcome back. I'm talking with cohere CEO Aiden Gomez, but how he convinces inherently risk averse enterprise customers to sign up for AI services. When I think about enterprise technology, it's very sticky that it will never go away. The first thing comes to mind is Microsoft Office,
which will never go away. That is the foundation of their enterprise strategy as office 365. Microsoft Office is a huge investment in OpenAI. They've got models of their own. A Kevin Scott has been on the show. They're CTOs in charge of AI. They're the big competitor for you. They're the ones in market selling Azure to enterprise. They're a hyper-scaler. They'll
give you deals. They'll integrate it directly so you can talk to Excel. The pitch that I've heard many times from Microsoft folks is you've got people in the field who need to wait for an analyst to respond to them. But now they can just talk to the data directly and get the answer they need and be on their way. That's very compelling. I think it requires a lot of cultural change inside some of these enterprises to let those sorts of things happen. You're obviously the challenger.
You're the startup Microsoft is 300,000 people. You're 350 people. How are you winning business for Microsoft? Well, I think they're a competitor in some respect, but they're also a partner and a channel for us. So when we released Command R and Command R+. Our new models, they were first available on Azure. I definitely view them as a partner in bringing this technology to enterprise. I think that Microsoft views us as a partner as well. I think that they want to
create an ecosystem powered by a bunch of different models. I'm sure they'll have their own in there. They'll have opening eyes. They'll have ours. It'll be an ecosystem as opposed to just only proprietary Microsoft tech. I think if you look at the story in databases, there you have fantastic companies like Databricks and Snowflake, which are independent. That's not a subsidiary of Amazon or Google or Microsoft. They're an independent company. The reason they've done so well is because
they have an incredible product vision. The product that they're building is genuinely the best option for customers. But also the fact that they're independent is crucial to their success. For that reason that I was describing where CTOs don't want to get locked into one proprietary software stack because it's such a pain and a strategic risk to their ability to negotiate. So I think the same is going to be true. It's even more important with AI where these models become
an extension of your data. They are the value of your data. The value of your data is that you'll be able to power an AI model that drives value for you. The data in itself is not inherently valuable. I think the fact that we're independent, folks like Microsoft, Azure, AWS, GCP, they want us to exist and they have to support us because the market is going to reject them if they don't. The market is going to insist on being able to adopt independence that let them flip
between clouds. And so they kind of have to support our models. That's just what market wants. I don't feel like they're exclusively a competitor. I view them as a partner to bring this technology to market. One thing that's interesting about this conversation and one of the reasons I was excited to have it with you is because you are so focused on enterprise, there's a there's a certainty to what you're saying. You've identified a bunch of customers with some needs. They've
articulated their needs. They have money to spend. You can identify how much money it is. You can build your business around that money. You keep talking at the market. You can spend your budget on technology appropriately for the size of the money that's available in the market. When I ask if it's a bubble, what I'm really talking about is a consumer side. There's these big consumer AI companies that are building big consumer products and their ideas. People pay 20 bucks a month
to talk to a model like this. And those companies are spending more money than on training than you are. They're spending more money per year on compute than you are. They are the leading edge companies. I'm talking about Google and OpenAI, obviously. But then there's a whole ecosystem of companies that are paying OpenAI and Google a margin to run on top of their models to go sell a consumer product at a lower rate. And that does not
feel sustainable to me. Do you have that same worry about the rest of the industry? Because that's what's powering a lot of the attention and a lot of the interest and all the inspiration, but it doesn't seem sustainable. I mean, I think those folks who are building on top of OpenAI and Google should be building on top of Co-ear will be a better partner. I'll leave that one up for you. You're right to identify that the technology providers focus. It might conflict
with its users. And you might find yourself in situations where I don't want to name names, but let's say there's a consumer startup that's trying to build an AI application for the world. And it's building on top of one of my competitors who is also building a consumer AI product. There's a conflict inherent there. And you might see one of my competitors sort of steal the ideas
or rip off the ideas of that startup. And that's why I think the Co-ear needs to exist. You need to have folks who are like us who are focused on building a platform to enable others to go create those applications and are really invested in their success free of any conflicts or competitive nature. That's why I think we're really good partners because we're focused. And we let our users succeed without trying to compete or play in the same space. We just build a platform
that you can use to adopt the technology. That's our whole business. Do you think it's a bubble when you look around the industry? I don't. I really don't. Just in terms of myself, I don't know how much you use LM's day to day. I use them constantly. Multiple times an hour type thing. How could it be a bubble? I think maybe the utility is there in some cases. The economics might not be there. That's how I would think about it being a bubble.
I'll give you an example. You've talked a lot about the dangers of overhyping AI. Even in this conversation, but you've talked about it publicly elsewhere. You've talked about you've got two ways to fund your compute. You can get customers and grow the business or you can raise money. I look at how some of your competitors raise money and it's by saying things like we're going to build a GI in the back of hell once and saying things like we actually need to pause development.
So we can catch up because we might destroy the world with this technology. That stuff to me seems pretty bubbly. We need to raise a lot of money so we can continue training the next frontier model before we've built a business that can even support the compute of the existing model. But it doesn't seem like you're that worried about it. You think that's going to even itself out.
I very much agree that is a precarious setup. The reality is for folks like Google at Microsoft, they can spend tens of billions of dollars on this and it's fine. Right. The amount of capital that it's like it's genuinely okay. It doesn't really matter. It's a rounding error. I think for startups taking that strategy, you either need to become a subsidiary of one of those big tech companies that prints money or do some very poor business
building in order to do that. That's not what cohere is pursuing. I agree with you to a large extent. I think that that's a bad strategy. I think that ours, the focus on actually delivering what market can consume and building the products and the technology that is right sized or fit for our customers. That's what you need to do. That's how you build a business. That's how all successful businesses were built. We don't want to get too far out in front of our skis.
We don't want to be spending so much money that like you say, it's hard to see a path towards profitability. Cohere's focus is very much on building a self-sustaining independent business. So we were forced to actually think about this stuff and steer the company in a direction that supports that. You've called the idea that AI represents existential risk. I believe the word you've used is absurd and you said it's a distraction. Why do you think it's absurd? What do you think
the real risks are? I think the real risks are the ones that we spoke about, which is like over eager deployment of the technology too early. People trusting it too much in scenarios where, frankly, they shouldn't. The doomsday scenarios and the terminator scenarios. I'm super empathetic to the public's interest in those scenarios. I'm interested in those scenarios because I've watched sci-fi and like, you know, it always goes badly. So we've been told those stories or decades
and decades. So it's a very salient narrative. It really captures the imagination. It's super exciting and fun to think about, but it's not reality. It's not our reality as someone who's technical and quite close to the technology itself. I don't see us heading in a direction that supports the stories that are being told in the media. And like you say, often by companies that
are building the tech, I really wish that our focus was on two things. One is like the risks that are here today, like over eager deployment, deploying them into scenarios without human oversight, those sorts of discussions. And when I talk to regulators, when I talk to folks in government, that's the stuff they actually care about. It's not doomsday scenarios. It's like, is this going to hurt the general public if the financial industry adopts it in this way or the medical
industry adopts it in this way? They're quite practical and actually grounded in the reality of the technology. Those stories are very exciting. And so that's why they dominate the headlines. I mean, the other thing that I would really love to see a conversation on is the opportunity. Like the positive side, we spend so much time on the negatives and fear and doom and gloom. I really wish like someone was just talking about what we could do with the technology or what we
want to do. Because as much as it's important to steer away from the potential negative paths or bad applications, I also want to hear the public's opinion and public discourse about what are the opportunities? Like what good could we do? I think one example in medicine, like apparently, doctors spend 40% of their time taking notes. And this is in between patient visits. You have your interaction with the patient. You then go off, you go to your computer and you say, so it's
okay, man, they had this. I remember from a few weeks when they came in, it looked like this. We should check this the next time they come in. I prescribed this drug. So they spend a lot of time typing up these notes in between the interactions with patients, 40% of their time,
apparently. If we could attach passive listening mics that just go from patient meeting to patient meeting with them, transcribe the conversations and just like pre-populate that so that instead of having to write this whole thing from scratch, they read through it and they say, no, I didn't say that. I said this and add that. And it becomes an editing process. We bring that 40% down to 20%. That overnight, we have 25% more doctor hours. And I think that's incredible. That's a huge
good for the world. We haven't paid to train doctors. We haven't added more doctors in school. We have 25% more just by adopting technology. And so I want to find more ideas like that. What application should cohere be prioritizing? What do we need to get good at? What should we solve to drive the good in the world that we want to see? There's no headlines about that. There's no one is talking about it. And I really wish we were having that conversation.
As somebody who writes headlines, I think one, there aren't enough examples of that yet to say it's real, which I think is something people are very skeptical of. Two, I hear that story and I think, oh boy, a bunch of private equity owners, a version of care clinics, just realized they could shove 25% more patients into the doctor schedule. And there's a real, a real animosity towards that. What I hear from our audience, for example, is they feel like right now the AI companies are taking a
lot without giving enough in return. And that's a real challenge. That's been expressed mostly in the creative industries. We see that generated. We see that anger directed the creative generative AI companies. You're obviously an enterprise. You don't feel it. But do you see that? That you've trained a bunch of models. You should know where the data comes from. And then the people who made the original work that you're training on probably want to get compensated for it.
Oh, yeah, totally. I'm very empathetic to that. Do you compensate where you train from? Yeah, we pay for data. We pay a lot for data. So there's a bunch of different sources of data. There's stuff that we scrape from the web. And when we do that, we try to abide by people's, like if they express we don't want you to collect our data, we abide by that. We look at robots.txt. When we're scraping code, we look at the licenses that are associated with that code. We filter out
data where people have said clearly don't scrape this data or don't use this code. If someone emails us and says, hey, I think that you scraped x, y, and z. Can you remove it? We will of course remove that and all future models won't include that data. So we don't want to be training on stuff that people don't want us training on like full stop. So like I'm very, very empathetic to creators and I really want to support them, build tools to help make them more productive, right?
And help them with their ideation and creative process. That's the impact that I want to have. And I really want to respect their content. Yeah. The other flip side of it is the same creators. Are watching on platforms they publish on get overrun with AI content and they don't like it. Right. So there's a little bit of a competitive aspect there. That's one of the dangers you've talked about. Right. There's a straightforward misinformation dangers on social platforms that don't seem
to be well mitigated yet. Do you have ideas on how you might mitigate just AI generated misinformation? Yeah. I like that's one of the ones that scares me a lot is the democratic world is vulnerable to influence and manipulation just in general. Take out AI like just democratic processes are very vulnerable to manipulation. If you're able to build yourself a platform where you can project ideas, I guess we started off the podcast saying this, people are the average of the last 50 posts
that they've seen or whatever. You're very influenced by what you perceive to be consensus. If you're like I look out into the world on social media and everyone seems to agree X then you're like, okay, I guess X is right. I trust the world. I trust consensus. Humans are we are pack animals, herd animals. Democracy is vulnerable and it's something that needs to be very vigorously protected. And I know you can ask the question about how does AI influence that? What AI enables is much more
scalable manipulation of public discourse. So you can spin up a million accounts and you can create a million fake people that project one idea and present a false consensus to the people consuming that content. Now that sounds really scary. That's terrifying. That's like this is a huge threat. I think it's actually very, very preventable. So for instance, these social media platforms, they're the new town square. And in the old town square, you knew that the person
standing on their soap box was probably a voting citizen alongside you. And so you cared a lot about what they said. In the digital town square, everyone is much more skeptical of the stuff that they see. You don't just take it for granted. We also have methods of confirming humanity. Like human verification on social media platforms is a thing. And we need to support it much more thoroughly so that people can see is this account verified? Is it now shaped person on the other side?
What happens when the humans start using AI to generate lies at scale? Me posting an AI generated image of a political event that didn't happen is just I think just as damaging if people believe it is thousands of robots doing it. What I was talking about was when you can have a single entity creating many, many, many different voices saying the same thing to present consensus. You can stop that by just preventing fake accounts. And just having with each account, there's a human verified
ID behind it. So you know it's another person on the other side. And that stops that scaling of millions of fake accounts. On the other side, what you're describing is fake media. I mean, there's already fake media. There's Photoshop. We've had this tech for a while. I think it becomes easier to create fake media and there's a notion of media verification. But you also, you're going to trust different sources differently. If it's your friend posting it, who you know in the real
world, you trust a lot. If it's some random account, you don't necessarily believe everything that they claim. And if it's coming from a government agency, you're going to trust it differently. If it's coming from media, depending on the source, you're going to trust it differently. And so we know how to assign appropriate levels of trust to different sources. So I think it's definitely a concern. But it's one that is addressable. Humans are already very aware that other humans lie.
Yeah. I think you're being very optimistic about that. I look at Facebook and it's full of tens of thousands of pictures of Jesus made of spaghetti and people are happily liking those pictures. And there's nothing about that. That seems like a person. I want to ask you one last question. It's the one I've been thinking about the most. And it comes, it brings us back to where we start. We're putting a lot of weight on these models, business weight, cultural weight, inspirational
weight, right? We want our computers to do these things and the underlying technology is these LLMs. Can they take that weight? Can they withstand the burden of our expectations? That's the thing that is not yet clear to me. There's a reason cohere is doing it in a targeted way. But then you just look broadly. And there's a lot of weight being put on LLMs to get us to this next place in computing. You were there at the beginning. I'm wondering if you think the LLMs can actually
take the weight that's been put on them. Take the pressure that's been put on them. I think we'll be perpetually dissatisfied with the technology. And I think that the technology will be continuously rising to the occasion. So I don't think there's some state. Like if you and I chat in two years, we're going to be like disappointed that the models aren't inventing new materials fast enough. We get us, you know, whatever, whatever. So I think that we will be always disappointed
and wanting more because that's just part of human nature. But I think the technology will at each stage impress us and rise to the occasion and surpass our previous expectations of it. But there's no point at which people are going to be like, we're done. Like we're good. It's not done. I don't think a question. I'm not asking if it's done. I'm saying, do you see as the technology develops that it can withstand the pressure of our expectations?
That it has the capability or at least the potential capability to actually build the things that people are expecting to build? Absolutely. I think it will. There was a period of time where everyone was like, the models hallucinate, they make stuff up. They're never going to be useful. We can't trust them. Never going to be useful. And now hallucination rates, you can track them
over the years. They just dropped dramatically and they've gotten much better. I think with each complaint or with each fundamental barrier, like all of us who are building this technology, we work on it and we improve the technology and it surpasses our expectations. I expect that to continue. I see no reason why. Why it should. Do you see a point where hallucinations go to zero? That to me, that's when it unlocks.
You can start depending on it in real ways when it stops lying to you. Right now, the models across the board hallucinate in honestly hilarious ways. But there's a part to me anyway that says I can't trust this yet. Is there a point where the hallucination rate goes to zero? Can you see that on the roadmap? Can you see some technical developments that might get us there? You and I have non-zero hallucination rates. Well, yeah, but no one trusts me to run anything.
You know, as I sit here asking the questions and you're going to see, yeah, like fine. But I'm saying computers, you're going to put them in a loop like this. You want to get to zero. No, I mean, humans, they misremember stuff. They make stuff up. They get facts wrong. If you're asking whether we can beat the human hallucination rate, I think so. Yeah, definitely. That's definitely an achievable goal because humans hallucinate a lot.
I'm like, you know, misremember stuff. I think we can create something extremely, extremely useful for the world. Useful or trustworthy. I think that's what I'm getting at. There's trust. And I think the amount that you trust a person varies, sure. Some people lie more than others. The amount that we have historically trusted computers has been on the order of a lot. And with some of this technology, that amount has dropped, which is really interesting.
And I think my question is, is it on the roadmap to get to a place where you can fully trust a computer in a way that you cannot trust a person. We trust computers to fly F-22s because a human being cannot operate an F-22 without a computer, right? If we're like the F-22 control computer is going to lie to you a little bit, like we would not let that happen. It's weird that we have a new class of computers where we're like, well, trust it a little bit less.
Yeah. Yeah. Like I mentioned before, I do not think that large language models should be like prescribing drugs for people or doing medicine. But I promise you, like if you come to me, Aiden, with a set of symptoms and you ask me to diagnose you, you should trust cohere's model more than me. Yeah. Like it knows way more about medicine than I do. Like there's just whatever I say is like going to be much, much worse than the model. And that's already true, like just today, like in this
exact moment. At the same time, neither me nor the model should be diagnosing people. But it is more trustworthy, right? Like you should genuinely trust that model more than this human with that use case. And so in reality, who you should be trusting is the actual doctor that's done like a decade of education on all this stuff. So the bar is here, Aiden's here, the model is, you know, slightly above Aiden, we will make it to that bar. I absolutely think. And at that point, we can
put the stamp and say it is trustworthy. It's actually as accurate as the average doctor. One day, it'll be more accurate than the average doctor. We will get there with the technology that there's no reason to believe why why we wouldn't. But it's continuous. It's not a binary between you can't trust the technology and you can't it's where can you trust it? Because right now in medicine,
you can't like we shouldn't like really rely on humans. But in other places, you can, right? Like when there's a human in the loop, it's actually just an aid and it's like this augmentative tool that is really useful for making you more productive and doing more or having fun or learning about the world. And so there's places where you totally can trust it effectively and deploy it effectively already today. That space of places that you can deploy this technology and put your
trust in it, it's only going to grow. And to your question about, you know, will the technology rise to the challenge of all the things that we wanted to do? I really deeply believe it will. Yeah. Well, and that's a great place to end it. Thank you so much for attending. I was talking to Dakota. This is really great. Yeah, thank you for having me. I'd like to thank Aiden for taking the time to join Dakota and thank you for listening. I hope
you enjoyed it. If you'd like to let us know what you thought about this episode or anything else, drop us a line. You can email us at decoderthoverge.com. We really do read all the emails. We can hang the app directly on threats on that reckless 1280. We also have a TikTok. For as long as there's a TikTok, it's at decoder pod. It's a lot of fun. If you like decoder, please share it with your friends and subscribe over here. Podcasts,
if you really like it, hit us with that five-star review. Decoder's Repression Verge, part of the box made of podcast network. Our producers are Kate Cox and Nick Statt. Our editor is Kelly Wright. Our supervising producer is Liam James. The Decoder Music is by Breakmaster Cylinder. We'll see you next time.