#184 - OpenAI's Voice 2.0 + execs quitting, Llama 3.2, To CoT or not to CoT?

I finished a PhD in AI from Stanford last year, and I now work at a generative AI startup. And as we've been saying the last couple episodes, Jeremy is busy with having a baby and not sleeping as of recently. So congrats to him, he'll be off for the foreseeable future. I'm not sure how long it takes to adjust to that, but presumably some time. Uh, so we do have a guest co host and it is once again, as he has done now

I doubt it. The congratulations is just Well, congratulations to any listener out there who's recently had a kid. I've, we're really rooting for you. Um, yeah. And so I guess my two line biography, um, I host the world's most listened to data science podcast. Um, it's called super data science and, uh, unlike the show. So it isn't a new show, really. It's, it's more of an interview show about two thirds of episodes have a guest. We do an episode every Tuesday and Friday.

And, uh, we also do sometimes on those Fridays, about half of the time, instead of having a guest, I kind of do a deep dive into a topic. So, um, a recent one, for example, I did a half hour on the O1 algorithm from open AI, because as you guys said, it's like, I think the quote from Jeremy was The release of the quarter, at least, and I absolutely agree. It's a big sensational thing.

I won't go into detail on that because you guys have already covered it, but uh, yeah, that's the kind of thing we do. I'm also co founder and chief data scientist at an AI company called Nebula, which is, yeah, automating white collar processes with AI. And yeah, I actually used something recently that happened. Andre is, I started doing some stuff on TV last year and I have a couple more things, which I can't publicly announce yet, but are in the works are going to happen very soon on TV.

It's pretty exciting.

Conversations about AGI and artificial superintelligence and how that's going to transform society in that episode, which was not, you know, that wasn't my intended really topics to cover, but we got deep into it and that incidentally, that is what at least one of these big TV projects is about. It's kind of, it's creating a show for the masses about how everything could dramatically change in the coming years as machines eclipse us with their intellectual capacity.

And yeah, Jeremy's been on the show a bunch of times as well. Uh, he was most recently on episode 545, it looks like. So people can check those out if they want to learn more about the great hosts of last week and AI.

I love it, but does not release consistently. That has certainly been true. Recently, I will try to look into hiring an editor because that is a major bottleneck, me having to do the edit and I have pretty, you know, specific requirements that I tried to get done. So it hasn't been, uh, quite as easy as, You would like it to be, but, uh, every way, like, let's hope, let's hope I managed to keep it a bit more consistent going

But then getting consistent revenue, like sponsorship, that kind of thing, that's another time commitment, which would eat into your ability to edit and release episodes. So it's a tricky, tricky situation to get these things rolling.

Uh, actually commented that it got a low score, suggested like a type of theme for the next one, type of music. Which actually is helpful because every time I'm generating a song, I'm just like, well, what genre should I do this time? You know, what haven't I done yet? So if you do have any requests for type of music for the intro or outro, feel free to send an email, comment on YouTube, one of these things, and I will certainly probably give it a try.

What was your prompt for the most recent one? Was that like David Bowie or something? It was like six. 60s psychedelic or something synth y, but 60s rock. Uh, I, I usually try like a couple and that one got something interested, interesting going on. It's, I have so many like drafts of songs that are just discarded. All righty, enough prelude, let's get into the news and we begin with tools and apps where we have some kind of exciting developments in the emerging space of conversational AI.

So starting out with OpenAI, they are rolling out advanced voice mode with more voices and a new look. So we've had this voice mode thing where, you know, we got GPD 4. 0. Oh, Omni. Now, quite a while ago, and in the introduction to that model, kind of a big deal about it was that it was conversational. You could speak to it, not just via text, but via your voice. And it was almost real time, right?

It was, uh, kind of, Pretty groundbreaking, I would say, as far as conversational audio interaction with an AI. And so with this new update, which is rolling out to paying customers, so Plus and Teams tier, they'll be getting custom instructions, five new voices. They call them Arbor, Maple, Sol, Spruce, And they'll not sure what those voices are like, but, uh, hopefully maple is really like sticky and rich. Yeah. Yeah. Yeah. And so that means that there's now, uh, nine voices that you can use.

And, uh, some other stuff going in here, uh, getting a new design now represented by a blue animated sphere, which is, I guess, cooler if, and just like the little, uh, music or audio notes that they add, and, uh, that's about it. Uh, they've been kind of slow to roll out to this feature. And so it seems that they are now. Gathering enough data to improve accents, to be able to expand voices, and so on.

And, uh, it appears primarily available in the U. S. This is not available in the EU, the UK, Switzerland, various other

can be supported by much smaller devices. You don't need to have a screen open in front of you. You don't need to slow down whatever you're doing to use your fingers to type. And so this is obviously the future. And yes, this advanced voice mode from OpenAI is right at the cutting edge in terms of capabilities.

end the demo with this really like gut wrenching video of beautiful things like soccer goals being scored right in front of you and you're like looking around the stadium and a woman tightrope walking and sharks and it just like they try to make like this really immersive and beautiful vision of the world and excitement and you're like wow it's so great to be alive it's it's amazing Even better with this headset on than being out in the world. Um, I don't know.

Anyway, I liked it a lot, but, uh, yeah, just kind of tangentially related to, you know, where if it's, it seems like this kind of thing, like augmented reality, like it's easy to imagine how in 10 years, maybe instead of having a bunch of screens around your desk, you can just have All of the space in whatever room you're in being occupied by screens. And I think the only real limitation right now is battery life because you can only do two hours on the Apple Vision Pro.

Um, and so you can like completely replace your office setup right now with that.

I really like these non human names that are like, uh, coming from nature. And according to the Wall Street Journal, Meta is paying these celebrities millions of dollars for their voices. So again, uh, interesting to me, I feel they've tried this before already, prior to the expansion with Llama on the, all these platforms. So quite an investment from Meta And I guess trying to get people to interact with their AI kind of offerings across these apps.

I can say I still haven't used anything and I use WhatsApp and Instagram and I kind of just ignored the AI on that front. I'm sure many people are on the same boat. So perhaps this is meant to

Like when you use cloth 3. 5 sonnet. And so, I don't know, it isn't very compelling. I don't, I don't It doesn't seem obvious to me right now that I need to be having a generative AI conversation in WhatsApp instead of in the chat GPT app or the cloud app, which is equally close to me in my phone.

And then another strategic difference here is that open AI, for whatever reason, has decided to try to not be partnered with specific celebrity names and to just have those generic tree names, even though. They obviously did famously get in trouble with Sky being too close to Scarlett Johansson and being sued by her.

But, um, but yeah, it's a different, it's an interesting strategy for Meta to say, okay, you know, we're going to deliberately partner with these celebrities to get their voices in and opening eyes seemingly except for the Scarlett Johansson blip. Deliberately avoiding that. So I don't know. I don't know. I don't know if I have like a, an explanation or something bigger to say that, but I want to call out that distinction

It's only available in English, but they are saying that it will arrive later with new languages. It's also only on Android. It will arrive on IOS later. But if you are using me. Android and, uh, are speaking English. You can speak to it pretty easily. There's like a new way form icon in the bottom right corner of the app, uh, or overlay of Gemini. So another example of, you know, this. kind of modality is making pretty rapid progress.

It's only like a few months ago that we even got demonstrations that this is possible. Now it's kind of rolling out wide.

And yeah, I thought, so I'm a fan of Gemini in general. I, it's great to see that they're making this progress on the voice front as well, just like the other big players that we've mentioned in this episode already, Meta and OpenAI. And yeah, I think the big thing here with all, for all of these companies is going to be, like you said, near real time. It's once this becomes Transcribed Actually real time.

And you can see with Gemini here, um, apparently I haven't used the Gemini voice functionality, but apparently you can interrupt the AI mid sentence, which is going in that direction, but it's going to be where you feel just like you and I talking on this episode right now, Andre, where it seems like everything that I'm saying can be, uh, responded to kind of really reactively, really in real time. And something that's going to be interesting is.

Once, so for a long time, you and Jeremy recorded this podcast audio only, but you went on video to record it. And that's because there's lots of extra information that we provide with our faces and our gestures that allows us to have an even better conversation. And so it's going to be interesting.

And I. I bet that some of these companies, maybe all of them have already been prototyping ways of having your webcam or your VR goggles be able to react to your facial expressions, because then you could have things like and things like this is completely speculative. I have no idea. But I use an Opal C1 camera and OpenAI recently acquired Opal, which is a hardware company that makes cameras. Why would they be doing that?

And it seems to me like there's a great opportunity there to be, because imagine you're looking at the outputs in a chat GPT conversation and you furrow your brow as you're reading it. Then it might, all of a sudden it might stop and say, Oh. I'm sorry. I realized I just made a mistake without you having to type or do anything. Like there is no technological reason why that couldn't happen.

And so it's probably just a matter of kind of like testing and feeling like you, um, have worked out the kinks and maybe, you know, ethical issues that could be associated with it.

Kind of a long title, but pretty informative. So both of these companies, Luma and Runway, kind of the leaders in the space of AI video generation, I would say, are now offering APIs. And APIs are what you can use programmatically. So instead of going on the Uh, user interface on the website and clicking buttons, right? Writing text.

You can now have some code that sends a request and this is how you would make, you know, another app or, uh, perhaps, you know, integrate it with your company's product, whatever. So you now have this API, it's connected to the latest version of dream machine. They say that it's priced at 10. 0. 32 cents per million pixels generated, which is a fun way to think about it, I guess.

And, uh, as that's, that's for Luma just hours prior to that runway has also launched the API where you actually fill out a form to get access to it now, versus GeoMachines API is available to use for everyone, I believe. And similarly has Gen3 Alpha Turbo with different pricing plans. pretty equivalent. So another trend where, you know, it's not that long ago that we started to see usable video generation, so to speak. And now they are trying to actually commercialize it with

Pages, similar to Word documents, but they can be expanded and collaborated on by team members. Kind of a reminiscent of artifacts from Cloud and some of these things like perplexity, which also allows you to publish the results of your searches. There's also another new feature, switching between web and work modes, that enables cloud. Copilot to tap into knowledge contained in all work documents.

And Gemini has some similar functionality, I believe, in Drive, where you can talk to it with awareness of some documents that are relative to your context. Last note, you know, presumably there's quite a lot things here, but interesting one, PowerPoint has a new narrative builder that creates a whole deck of slides from a prompt using.

Co pilot, including transitions and speaker notes, which, uh, for those working in, uh, the industry, industrial world in the corporate world, it might be kind of exciting.

So this is designed to solve puzzles, math problems and coding challenges. It's a feature currently in beta available to paying perplexity pro users. And currently very limited to just 10 queries per day. Probably because 01, uh, OpenAI's model limits your API access, uh, and it actually does not integrate with search. So this is essentially, it seems like kind of a connection to 01, just a way to use it indirectly.

Uh, but Perplexity has integrated various models pretty rapidly, and this is another example where. 01 just rolled out and Perplexity now has this in their tool.

I, I teach a calculus for machine learning curriculum, and I was taking some of the relatively advanced, it's an introductory calculus course really, but I was taking some of the more advanced questions that I have in that curriculum, like partial derivatives, and I was copying and pasting them into And it absolutely knocked them out of, out of the park in a way that you would never even think that you could try that with GPT 4. 0.

Like if you tried that with GPT 4. 0, like you know that the next token prediction is just, it's, it's an estimate. It's not doing math. Um, it's not going to be checking for mistakes. And so the odds with even a moderately complicated partial derivative calculus question of you getting to the, Correct answer are very small with GPT 4a. But with O1, every time, every exercise that I put in, integrals two worked really well.

And the way that it, uh, expressed the answers as well was really impressive. If I had a student in class that was like, That expressed it the way that 01 did, I'd be like, wow, you should be teaching this class. Um, and so it also helps with, uh, with understanding problems. So, um, with these kinds of things, like the partial derivatives, the integrals, 01 would think for about 10, 12 seconds. But I tried a much more complex math problem that I candidly I didn't understand.

I didn't understand this question But it took about 90 seconds processing To the answer and then provided me with a response after that processing That allowed me to actually even understand what the question meant and to get to the right answer as well So super impressive also tried to There's lots of coding tasks that I expect models like gpt40 and clod 3.

5 sonnet to be really great at and then I So I it took me a while to come up with an idea for something that I wouldn't expect clod or gpt40 to get correct and eventually what I thought it was creating an interactive website that is a neural network where I can hover over nodes in the neural network and get the bias at that part of like at that node for that neuron in the network. And if I hover over an edge in the network, it shows me the weight of that connection in the neural network.

And so I was like, that sounds pretty complicated. And you know, it was a long prompt with lots of details. It didn't a hundred percent match Get it right like there were things like I asked for arrows going from left to right to show forward propagation through the network and It wasn't rendering arrows. It was just rendering straight lines.

But other than that, it nailed all of it the interactivity in HTML it was a huge amount of code and you know I mean, so it shows the potential for, yeah, this is, I mean, in a nutshell on a lot of it, like if you want to get like this, having as Jeremy described it, having this other, this additional way of scaling. So we already had, you know, scaling laws associated with making these LLMs larger and larger, scaling the amount of data. Now you can also scale inference time.

And so an obvious extension, which Noam Brown, one of the researchers working on a one at open AI, he wrote in a really great Twitter post about how this kind of Processing now can be, can be expanded. You can just add inference time instead of thinking on the seconds scale, you can think on minutes, hours, days, maybe weeks. And then you could put in really complex questions like solve cancer and come back six months later and, you know, see how it's doing.

I'm being a bit reductive, but there's a huge amount of potential here. And as the cost, like right now, the cost is obviously very expensive, but Like everything else in computing, the cost is going to continue to decrease at crazy rates. We're going to figure out software cleverness to make it much cheaper at inference time and hardware is always getting cheaper as well. So we already have for specific use cases like physics, chemistry, biology, 01 demonstrates PhD level capabilities.

And so even if we just were at today's level of capability with L1, you know, that it's going to get cheaper in the coming years. And so that's, then you theoretically have basically limitless number of PhD level, hard scientists all around the planet, just crunching through problems. That is a serious game changer.

And then on top of that, when you factor in the things like longer inference time, more cleverness around the way that they train these models, this isn't just going to be PhD level in hard sciences. This is going to be. It seems obvious to me more than PhD level, like serious, like, you know, lesser level or beyond in, you know, any imaginable quantitative subject.

And so that's, and as that becomes really cheap, then, you know, having that it's, it's effectively unlimited intelligence, far beyond human capabilities. Like we're already, We're looking at that now, these kinds of things that seem like science fiction just a year ago,

There are some challenges to just getting all want to be better, but definitely exciting. And from my personal experience and your experience, you know, believe the hype, so to speak. And moving on to applications and business, once again, as is often the case, we have some stories related to open AI that are not related to all one. But instead, some drama with their, uh, executive team and their corporate structure.

So when you story here, which just came out today, uh, dramatically titled open AI execs must quit as company removes control from nonprofit board and hands it to Sam Altman. I will say that's being a little bit overdramatic, but, uh, is also not entirely wrong. So the. Big part of this kind of the first wave, at least what I've seen was a company CTO Amira Moradi announcing her departure. And she has been there for six and a half years. So a pretty significant move.

Uh, she was for some background and involved in some of the drama with Sam Altman being removed from his position as CEO last year, although she was the interim CEO. Yes, she was the interim CEO. She expressed some concerns, it sounded like, although I don't believe she was directly pushing him out. Regardless, she is now announcing her resignation. She's saying that she wants to pursue other opportunities, explore options. So there's not, you know, any huge drama here.

But in addition to her leaving, the VP of Research and the Chief Research Officer Officer are also stepping away, which does seem a little significant to that. There's been these departures and this is falling on some notable researchers leaving just earlier this year. And all of this is falling on the heels.

We don't have concrete Uh, details yet, but there is more and more details or, you know, rumors coming out of this plan move away from being a nonprofit to being basically entirely for profit, more traditionally and reportedly also Sam Altman is going to get equity in opening, I kind of get basically some amount of control and some amount of, uh, monetary, uh, I guess benefit from being the CEO.

So, yeah, another chapter in the exciting and quite, uh, dramatic history of OpenAI's governance, uh, corporate structure,

It does seem, it does seem interesting because if you think about how well the company's doing commercially and in terms of their brand, I mean, I don't know more. Do you think more than half the time on this podcast last week in AI, the opening story is an open AI story. Yeah, it's wild. Like it's there. There's so much at the forefront of what's happening in AI period that you'd think people would be. Clinging to that more than ever before.

You're like, wow, this thing that we've been working on, it's really working, but simultaneously people are leaving. So yeah, these things like going for profit, uh, you know, becoming closed AI. Um, Yeah, I guess that changes, I guess,

So, you know, it, it was a very gradual shift in some sense, away from being a more open, just research. Company to now being a massive, you know, multi billion dollar, uh, company business that Seems to be shifting very much away from being this R& D lab with a focus on AGI having like, you know, benefit for all, uh, cap profit, all of these kind of weird things that are, uh, inspired or motivated by this idea that they are going to get, uh, AGI. Again, we don't know the full details of this.

Supposed move to for profit and so on. We don't know if the departures are directly related, but You know, as ever, interesting stuff seems to be

My memory of it was that Sam didn't have equity. And that always was weird to me. The CEO didn't have equity. And now it would be 7 percent supposedly is what is being reported. And that's a lot. That's a lot. Cause often startups kind of reserve about 10 percent of their capital for all employees. Including the CEO. So 7 percent is a huge amount.

Leadership changes are a natural part of companies, especially companies that grow so quickly and are so Demanding. I obviously won't pretend it's natural for this one to be so abrupt, but we are not a normal company. And I think for reasons Mira explained to me, there's never a good time. Anything not abrupt, we have leaked. And she wanted to do this while OpenAI was in an upswing. Make sense. So there you go.

Yeah. Like let's not, uh, you know, make it sound like OpenAI is definitely in some turmoil,

And in fact, if anything, I mean, I realize I'm just creating like, you know, I'm generating drama here. But, you know, even that, like to have all of these people that you kind of expect, you know, everyone's kind of watching, what is Greg Brockman going to say? What is Sam Altman going to say? And then they all have these like really nice things to say. It seems like it could be a bit, uh, manufactured, I guess. But yeah, again, completely speculation, maybe there is an injury on that at all.

And, uh, you know, the committee will still be responsible for things like safety reviews, including their review of things like O1 and will continue to receive regular briefings from OpenAI safety and security. teams. So for me, hard to read fully into what this implies. It sounds like maybe it's becoming more independent, which is something you'd want, right?

You don't want Sam Altman to be on a committee that is overseeing and evaluating how safe something is that you may want to release for commercial interests.

Now they have partnered with oil producer Aramco to build a giant data center in Saudi Arabia. This is going to have apparently 19, 000 language processing units initially, and Aramco will fund the development that is expected to cost in the order of nine figures, according to an interview with the CEO.

They say the data center will be up and running by the end of the year, and could later expand to include a total of 200, 000 language processing units, uh, which would make this a massive, massive AI inferencing center, right? We, we've heard, uh, Elon Musk, of course, saying that he'll have 100, 000 GPUs up and running at their, uh, super AI inference center. So another one being worked on here.

Um, but you know, there, there are efforts to have, you know, more and more tariffs and sanctions going both ways between those two groups. And Saudi Arabia is one of those countries that kind of sits in the middle. Um, it. Freely does trade with both the West and the other axis.

And so, yeah, it's an interesting, yeah, it's, it's an interesting play and it's interesting how, at least for now, there doesn't seem to be any resistance from say us politicians to be selling 200, 000 language processing units to Saudi Arabia, which obviously is a critical partner to the U S in the Middle East. Um, but yeah, they're also a critical partner to China. So it's, yeah, it's an interesting, uh, I don't know a lot about geopolitics, but yeah, But, um, yeah,

So basically some real advanced people who were involved in some of the pivotal technology and progress in AI image generation. The company is pretty new, actually pretty fresh. Have previously raised 31 billion. And it sounds like they are now going to raise a bunch more and get to the 1 billion valuation AI startup club, which is, you know, not as prevalent, not as easy to do nowadays as it was maybe a year ago. So Black Forest Labs, clearly a lot of excitement about

They're introducing the D7, a semi humanoid robot with an 8 hour battery life, 10 kilogram lift power, Another one that is seemingly going to be somewhat affordable, that is practically usable, potentially, it was unveiled by them in May as part of the long term strategy for service Robotic sector. So potentially seeing this in stores, et cetera. Final notes. What does semi humanoid mean? And that means to be, to be clear, it means that this has no legs.

So basically it drives around on a wheeled base, but the top of it, it has a torso arms and a face. It's kind of, uh, it's a Roomba with a torso on top, which I guess we haven't seen many of our people do. So perhaps the semi humanoid

But that being able to go up and down stairs comes at the expense of a lot of extra compute and probably also power required just to stabilize. Whereas when you've got wheels and it's just supporting a torso, that's going to be great for battery life. And it's going to be useful for a lot of situations because now you can be doing manual manipulations on.

you know, countertops or, you know, at a bunch of different levels that humans can access things at with their own arms, um, without, yeah, all that extra expense and complexity and presumably battery usage associated with legs.

It will be available in beta to select U. S. sellers and it is You know, an ongoing demonstration of Amazon trying to include generative AI. They have introduced uh, AI powered shopping assistant named Rufus, a business chatbot Q. They have introduced like a Believe a thing that helps people who leverage AWS understand how to do this. This is yet another tool in that, uh, trajectory.

And apparently more than 400, 000 of Amazon's third party sellers have used its AI listing tool, which is, uh, in an increase, uh, a recent increase from just 200, 000 in June. So, you know, maybe third party sellers will get a lot

Rufus, the shopping assistant, business chatbot queue. Now, Amelia, it seems like that kind of, while I totally get how under the covers these are, this is different infrastructure, different model weights makes a lot of sense from a data science development perspective. But from a marketing perspective, I think it's pretty confusing. And I don't know, like I've seen Rufus show up in my Amazon shopping experience recently, and it's not something that. Like just like having meta AI in WhatsApp.

I'm like, ah, I'm good.

Now you can give it images, and they say that you can use it for various AI applications. Like understanding video, visual search, things like that. There are, uh, two vision models. So there's an 11 billion parameter vision model and a 90 billion parameter. And along with that, Llama 3. 2 comes with two lightweight text only models with 1 billion parameters and 3 billion parameters. Somewhat similar to what we've seen from Microsoft with PHY, what we've seen from Google with Jemma.

These are condensed models that appear to be working really well. They seem to be condensed from the very big Lama models into these relatively small language models that are still capable of a lot. This came along with a bunch of announcements actually that they've demoed. So they also announced their Ray Ban Metaglasses get more feature. They demoed live translation from Spanish to English. They have a. prototype augmented reality thing, which of course will integrate AI, presumably.

Uh, but certainly having an open source model that is very good and now capable of dealing with images is a bit of a game changer in the sense that Llama 3. 0 was already very important to the AI ecosystem as a large language model that is at least sort of competitive with the frontier models. And now we can do images, You know, it makes it even more of a useful tool for people who want to build something about relying on opening. I

And so I'm deeply grateful. You know, we, we are able to have our own proprietary models running on our own infrastructure. At least in terms of training, an almost trivial cost, it could cost hundreds of dollars, thousands of dollars to create really well fine tuned models for the tasks that we need them to. If we had to train, if we had to do the pre training, it costs hundreds of millions of dollars that MED is investing here.

And so that also then ties into, You know, we talked about all these earlier, earlier in the episode, we were talking about all these new voices being added in and meta trying to push us as consumers of these free products into using these AI tools.

And part of why that push is happening is because executives and meta Mark Zuckerberg would like to be able to show to investors that they're getting some kind of return that there's, you know, an increase in engagement in Instagram, say, as a result of these billions of dollars being spent on open source research.

So OVIS stands for Open Vision, and they introduced some kind of new techniques for having the architecture for multimodal large language models. They have a full paper out on this stuff, and this is an update on their codepot. So we're adding this new version 1. 6 released under an Apache license. Uh, and they have the model out. We have a demo out, and it, of course, is better. It is trained on a larger, more diverse, and higher quality data set and also has instruction tuning.

So it seems to be performing pretty well on various benchmarks, uh, beating out other multi modal large language models, uh, out there on Just about every kind of benchmark, including Quen 2VL, which we covered last week, probably not beating Lamar 3. 2, but who knows, might be similar. So, exciting week for multimodal large models, for sure.

LLMs, which is really straightforward because we've got language going in and language coming out, even if that's a programming language, okay, still language, but then as these became multimodal, sometimes then they get described as a foundation model. which isn't as widely used a term and isn't disambiguous. And so I like this MLLM.

So you're, and I also like part of what I like about that is that even if it's multimodal, even if it's has vision capabilities, say as in the case of OVIS here, It's still relying on language, um, at, at, at a, um, at an abstract level in order to be able to do the vision capabilities that it has. So the, the language capabilities of the LLM enhance the world model that the vision model has to work with, that the vision capabilities have to work with.

Um, and, and, and a complimentary in that sense.

So, uh, there's potentially, uh, you know, you can scale up, train on more images and on more texts and, you know, you might get better overall results. Moving on to research advancements, we have a couple papers. The first one is to COT or not to COT? And the result of that question is chain of thought helps mainly on math and symbolic reasoning. So COT is a chain of thoughts. We've mentioned it quite a few times.

So to recap quickly, it is just telling your model, you know, think, a bit and, uh, first enumerate your, uh, chain of thinking, basically think through the problem and then give me the answer instead of just jumping to the answer. And there's been a lot of research on this, a lot of kind of known, uh, ways of using it. Of course, GPT, uh, not GPT 01 from OpenAI has chain of thought baked in and trained to do.

chain of thought or you could say reasoning that is certainly related to chain of thought. In any case, this paper is looking at whether chain of thought prompting is actually useful and makes VL1 better and what they have found from quite a large analysis of over 100 papers is that It does help on math and symbolic reasoning, but on other tasks like common sense, reasoning, text classification, context aware QA, it is, let's say, less of a difference.

It doesn't give you quite as much of a jump in performance. So this is, let's say, maybe more of an empirical paper, right? They're just demonstrating. a result of a lot of different evaluations of using this. Perhaps not fully surprising that chain of thought and kind of enumerating your thinking is mainly useful for math and symbolic reasoning, logical reasoning, these kinds of things that require you to talk through kind of a problem.

But another nice finding also for trying to get a better sense of how alarms work. I

Nobel prize winning economist was famous for decades for creating lots of research, particularly with someone named Amos Storky about how human brains work. And one of the big conclusions. that they had, which is right in the title of the book, thinking fast and slow from Daniel Kahneman is that we have these two, uh, systems of thinking. And so you have the fast system, one thinking fast and slow. So the first one is the fast one system one. And that's like your intuition.

That's what I'm doing right now. Words are just coming out of my mouth. They're just spilling out. I'm not planning what I'm saying. It's just happening. And that's like GPT 4. 0. And so on. tasks where you're generating an email or you're editing the copy of a document that you wrote. That kind of problem can be handled very well by that kind of system one, just spewing words out without thinking ahead, um, that GPT 4. 0 can do.

And so in the O1, uh, research announcement a couple of weeks ago, there They did comparisons across different subject areas and on things like I just said, like writing an email or editing text, GPT 4. 0 might even do better than a one, or they're at least comparable in human evaluations.

They do about, you know, they perform about 50, 50 on human evaluations, whereas it's tasks that leverage your slow thinking the system to in that thinking fast and slow, where, you know, when I, uh, Uh, when I'm tackling a math problem that is, you know, not something that I've already become adept at, I need to break that down into parts. I need to spend time just staring with pencil and paper at the problem. And it's those kinds of problems where you have to stop and think.

That chain of thought, Types of systems like oh one significantly outperform. And so that's kind of my, you know, when I'm thinking about a task that I might need an LLM for, I'm like, you know, is this a system one, uh, kind of task, in which case I'll probably go to cloud 3. 5 sonnet today. Whereas if it's a system two kind of task, like writing some very sophisticated code, doing some, uh, doing math for sure.

Um, if I had physics problems regularly, like maybe Jeremy would, uh, he's probably just always sitting around at home doing physics problems, right? Um, then, you know, in those kinds of scenarios, I would go to O1 right away. Exactly.

And it makes sense if you are able to do kind of a quick common sense response, or if it's just needing to know some knowledge. There's no need to plan ahead, as you say. And next paper, quite relevant on a related topic, and the title is LLMs Still Can't Plan? Can LRMs, a preliminary evaluation of OpenAI's O1 on plan net. bench. This paper is using this, uh, notion I haven't heard before of a term, large reasoning model, LRM.

Not sure if it has been used before, but, uh, they do say that O1 or strawberry as what is codename. According to them, it's claimed to be a large reasoning model designed to overcome limitations of LLMs. They show in this paper that it does represent significant improvements on sort of classical planning tasks, but still isn't, you know, significantly, capable of doing long plans of, let's say, going on to 10, 12, 14 steps. So how do we do this? What is planning in this context?

What is plan bench? The idea here is that there's in AI, a sort of, whole category of problems known as planning problems, where essentially you have a goal state you want to get to represented in some sort of set of variables. So one example that they are working with in this paper is this block world problem where you have a set of blocks, they are stacked on each other or kind of in various physical configurations. You have a set of actions like picking up a block.

On stacking a block from on top of another block, putting down a block and stacking a block on top of another block. And so you may have a set of configurations for the blocks that you want to get to from an initial set where you can put them down. You need to do this set of actions, picking up, stacking, unstacking, one by one to get there.

And there's a whole set of algorithms going on to the very beginning of AI and robotics, like PDL, Shaky from Stanford, that solve this kind of thing, that find you the sequence of actions that get you from a state to another state. And you can do this, uh, you know, if you have exact actions, you don't need. machine learning, even you can just do planning.

So here they evaluate 01 to see if it can give an, a description of a state and some actions it can take, whether it can generate a valid plan. And it certainly does perform much better than GPT 4. 0 and just about any other language model, but.

As you increase the path length, uh, going to again, 12, 14, 15 steps, it still goes to 0 percent correctness on this context of this kind of block world type of problem where it's just a few types of actions and a very specific state that you want to get to unsurprising. In some sense, this is comparing you.

Soft reasoning, so to speak, neural net reasoning to something that is much more algorithmic planning in this sense is essentially doing search through a sequence of things that is kind of a playground of straight up algorithms, straight up like search code routines, and not so much neural

Uh, so it can simulate the world. And that's what a world model is. It's essentially the ability to predict what will happen if you take a certain physical action. And they trained in simulation and in the various techniques to get a state of the art world model quite a bit better. In actual context with, uh, robot handling of, let's say, clothes or individual objects, opening doors, things like that, that enables it to do pretty reliably in novel environments.

So exciting if you're a robotics person, if you are excited about the humanoid progress we've been seeing one more note, they have also launched the one x world model challenge, where they are incentivizing more progress by offering over 100 hours of video data pre trained model and having a cash prize for people able to even provide improvements on top of that.

Um, of an AI system to be able to explore, you know, cause it's very expensive to be actually out in the world exploring. Um, and so, yeah, if you can do that virtually somehow, like this appears to do, then yeah, could really speed robot training up.

And so this includes about a hundred inputs from the patient's medical record, including vital signs and a lab test results. It can alert doctors and nurses to patients who are getting sicker or require intensive care. or are on the brink of death, uh, requiring intervention.

So according to the study, they looked at more than 13, 000 admissions to the internal medicine ward and compared that to admissions to other subspecialty units and seems that it definitely helped in a context of the internal medicine

And if you think about this as an early iteration on this system and, you know, stretching forward, when you see these kinds of systems work well, like we're seeing here from chart watch and you say, okay, we have a hundred inputs at this kind of level of granularity. We could be recording so much more data in hospital systems and training much, you know, more and. More and more better, more and more better AI systems. And so this is like just the beginning.

So to already see this 26 percent drop in unexpected deaths with what is probably a prototype working on a relatively limited data is really exciting. And I think it kind of ties into a world that I think we will have in our lifetimes where a huge amount of data is being collected and monitored.

not just in emergency rooms, but in, in your bedroom in your home and, you know, allowing us to have a really great sense of our health and get these early warning systems on, you know, people being on the brink of death and interventions being able to happen.

And now to policy and safety, we are going back to a topic we've been talking about more and more, which is how will you actually power all of these data centers that require a whole lot of energy, much more than the grid has been used to supplying to data centers. And this next story is providing one. Answer. Apparently, the Three Mile Island nuclear plant will be open to power Microsoft data centers. Three Mile Island is a bit notorious.

It was the site of the worst commercial nuclear accident in U. S. history. And it seems that, uh, there was Power purchase agreement signed with whoever controls this to enable this, uh, nuclear plants to provide energy. They have an agreement for 20 years and the plant is expected to reopen in 2028 and renamed to Crane Clean Energy Center. This is reopening, to be clear, From it having been shut down in 2019 due to inability to compete with cheaper energy sources.

So this is not sort of like reopening a fully shuttered plant from decades ago. This is more of Microsoft investing to get another source of energy, uh, in addition to You know, I

There's some countries like Germany, That are really opposed to having nuclear energy, but it's a great part of an energy mix alongside solar and wind because it can provide lots of, you know, solar and wind. You can't always guarantee you're going to have sunshine or wind. Um, you know, you can have batteries, uh, and those are getting better and better as well.

But having nuclear fusion as your backup, as opposed to say, oil or gas, uh, generators is obviously way better for the environment, at least in terms of carbon dioxide. Yes, with nuclear fusion, you have nuclear byproducts, but we're pretty good at managing those, and we're pretty good at managing risks with nuclear power plants as well. Um, you know, newer generations of power plants don't have any, uh, any record of issues like we did with Three Mile Island.

include AB 2655, which would require large online platforms to remove or label deceptive and digitally altered election related content and provide mechanisms to report this content. AB 2839 expands the time frame in which entities are prohibited from distributing deceptive AI generated election material. So that's presumably meaning you know, in the context of an election, when, uh, are you meant not to, uh, do this kind of thing.

It also expands the scope of existing laws to prohibit deceptive content. Last up, AB 2355 mandates that electoral advertisements using AI generated or substantially altered content future, uh, disclosure. So there you go. A whole, you know, a little set of bills related to AI altered content for elections. We've covered, you know, not too many stories related to deepfakes and elections, but there have been some. So perhaps looking ahead at emerging trends.

It makes sense that there is actual, uh, legal precedents being established here for this kind of thing. I'm curious, uh, John, you are a podcaster, you have a lot of data, were you thinking of doing a digital replica at

So yeah, it's something, it's one of those things that kind of feels like. Like I always kind of feel overwhelmed by things that are going on and I'm like, Oh man, this seems like a lot to do here. You know, you're, you're then talking about, you know, whole new YouTube channels and podcast RSS feeds. But, uh, yeah, a lot of potential listeners out there that you could be making an impact on. So yeah, definitely something. To explore here.

So this is A little bit of a funny story, but I guess also a serious story. Uh, apparently the Federal Trade Commission has taken action against the startup DoNotPay, which was advertised as the world's first robot lawyer. And the FTC found that this company DoNotPay conducted no testing to verify if its AI chatbots output was equivalent to a human lawyer's input. level and did not hire any attorneys to validate its legal claims. So do not pay is paying this fine of 195, 000.

It doesn't seem huge. And, uh, they are, uh, there's a 30 day public comment period, presumably. And they have also agreed to inform consumers who subscribe to a service between Uh, the last couple of years about the limitations of these features, uh, they are also prohibited for making baseless claims, uh, related to the services being able to substitute for professional lawyers.

There's 193, 000. Nice, nice.

The tool is in beta and available to a small subset of creators, but presumably will be expanded later on. And apparently this tool is powered by Snap's own foundational video model. So, uh, yeah, they are integrating direct AI video generation for anyone to use. And as a result, they are going to use icons and context cards to inform users when AI is used to generate content and include a watermark for any AI generated video.

And moving right along, there's basically an equivalent story where YouTube Shorts, which is similar again to TikTok and Instagram, will be integrating Veo, Google's AI video model. And this was just an announcement. So creators there will be able to integrate and use AI video generation and some of the other features of Veo to edit, to remix, uh, and, uh, generate six second long extend alone video clips. So video generation coming to all sorts of platforms,

And they are going to try and see if they can replace VFX artists to create. backgrounds and special effects, for instance. Uh, you know, very preliminary. They're gonna see if this can be used in pre production or post production, uh, but, uh, seems to be a bit ahead of others in the sense of trying to train, uh, actually custom models on their own data. Alrighty, well, that's it.

We are done with another episode that hopefully will come out just a day or two after we record this one without any more delays. Thank you for listening as always. Uh, thank you for commenting, reviewing, and maybe subscribing on the last week in ai substack. As always, we appreciate your views, your comments, and so on. And we appreciate you, John, for filling in for Jeremy and being a great guest co host.

I'm sure everyone who's listening appreciates it, but it deserves highlighting. I mean, from curating the amount just of stories that you've got to pass over to then decide and curate on the list that you cover, organize it, make sure there's no duplication. And that's just before you start recording. setting up recording slots, doing the recording, doing all the post production. It's wild how much you do every week on the show, Andre, so I appreciate it.

And I'm sure all your listeners out there do.