#152 - live translation on phones, Meta aims at AGI, AlphaGeometry, political deepfakes

I finished my PhD focused on AI last year -- before I said it was earlier this year, but now it's last year -- and I now work at a generative AI startup.

absolutely do it. And you can also reach out to me on I'm on the Twitter or the X or whatever it's called now, but yeah.

I guess we do go pretty fast, so maybe it goes well together. I'm not sure. And there was another, fun, cool review by George, who just is very nice, says we are supposedly fun and fast, which is, we didn't try for it, but we'll see. Well, we have a lot of news this week. There was, surprisingly, quite a lot to happened. No major sort of world changing news, but a lot of kind of individual, somewhat significant bits and pieces.

So we'll be moving fast. And we do hope you keep up and it makes your run more enjoyable. So starting with tools and apps with her story is that Samsung's latest Galaxy phones offer live translation over phone calls and texts. And that's this story. In the S 24 line of smartphones. There will be the ability to receive calls in a language people don't speak, and receive a live translation of the call, both audibly and on screen.

And this is, following up. Also, meta has a similar feature in their meta smartglasses of live translation. If you're talking to someone in person, actually, we have these speakers in their glasses that do live translation. So received a significant, you know, to start getting hardware that does offer live translation from any language or not any language for so many, but for many languages, too many other languages.

Language barriers is really going to start becoming less of a thing as feels in this next year.

They're allowing people to pick different communication styles and have them like preprogramed into their, actually in the into the text version of this too. There is a text version of this translation service as well. So you can let it know that, hey, like let's get into a casual mode or, or whimsical mode or whatever. So kind of an interesting meta parameter that you can, you can tweak there.

I think one of the most interesting things, though technically about this is just the blazingly fast inference speed inference time that you need in order to do real time translation. Right? Because like in order for it to be natural, it's basically got to like translate your words right as they're coming out of the oven. And that requires very low latency. Apparently it's all happening on the device too.

So and this is for privacy reasons. So all you want, all the computations associated with the translation to happen on the phone itself. So not beamed to some central server. But that you know, that's really interesting. So all this blazingly fast translation, all these inferences are being done on the edge device on the phone itself. So I think a really interesting, question is what does the back end look like? Like how are they pulling this off at speed.

This is, I think, the fastest live translation thing I've heard about. Like it seems like it's pretty much instantaneous. So I think a pretty important phase transition in just like the effectiveness and market ability, market readiness, I should say of, of translation.

And if, you know, having this super fast model that's a slimmed down large language model can still handle very robust translation. And just to be very concrete, they offer a 13 languages starting out. So it'll be Chinese, English, French, German, Hindi, Italian, Japanese, and some other ones, a lot of a bigger languages in terms of, I guess, population seem to be covered here. So yeah, pretty exciting, I would say.

As far as news of on device AI and, the sorts of things we'll get in our phones and in the near future.

And you can think of it as a sort of AI smart assistant in your hand, which you can tell to do things. And the pitch is it would just sort of figure out how to accomplish whatever you wanted to accomplish without you having to scroll and type and

all that sort of thing. And yeah, with this knowledge that they'll be providing live info from perplexity, basically what that means is you can now ask it any question, and it will be using this sort of advanced, AI driven search and perplexity offers to be able to apply to any query, kind of a more advanced version of what you might have with, existing assistants like Siri, where you can ask it various questions and they can try to answer using all the techniques like knowledge graphs.

So apparently the first 100,000 rabbit R1 purchases are going to come with one year of the Perplexity Pro subscription. So it's sort of like a, you know, it is a deeper business partnership. Though that plan also includes access to newer LMS that include, like GPT four, and it's normally 20 bucks per month. So, it kind of kind of cool. I think at this point, this whole rabbit thing is really taking off. I, I guess was late in the game. I, I caught on to the announcement that came out.

I think this was the week that I was in DC that I was traveling. But, but yeah, I mean, you know, it's like this, like little Tamagotchi type thing. Maybe that's an analogy only millennials would get here, but like, did you play with Tamagotchi? I think you're.

So it'll be very interesting to see once people do get their hands on this and they type in whether that in practice is something that is a game changer or people will start having all the time, or if people just, you know, keep using their phones that already can do a lot of this. So yeah, we'll see.

this. But, and then replace it with just AI optimized stuff is still kind of interesting because we do have our phones, and maybe I just need to get a rabbit run and see how it works. Is it like the iPhone for AI or is it just, you know, another another fad?

Yeah, and a smaller feature is being thrown in there, but does show that, AI is being pushed throughout Google's, products and in various ways.

At least where lately is like last six months. And, this is sort of an interesting and interesting except for them. It's an obvious next step, and it's technically obviously very necessary. But, kind of interesting to note that it has has been a little while for them to, to be getting into this space.

It will automatically identify and label clips as dialog, music, sound effects, or ambient noise. And besides that there will also be automatic resizing of waveforms, and various kind of little adjustments, updating colors for clips for better visibility and stuff like that. But, I guess one last thing to note is the enhanced speech feature, which will improve the clarity of poorly recorded dialog. And will presumably be pretty heavily leaning on AI for that bit.

They initially they started off in San Francisco and now they're kind of. Yeah, looking for a license to expand out into into LA. San Francisco of course, like your default starting point for most tech things, just because that's where the companies are founded. And, so, you know, migrating to each new city has a bunch of policy and regulatory hurdles to factor in. It also has a bunch of new risks because the

cities can look different. So you need to kind of patch some some edge cases that can come up with your self-driving cars. So anyway, sort of interesting that this is now happening. They have a permit right now to operate 250 robotaxis in S.F. in San Francisco. And at any one time, they're operating about 100 of them. So, you know, unclear whether it's going to be at the same scale. But, this kind of interesting that we're finally starting to see this broader rollout of self-driving cars.

So yeah, Waymo seemingly continuing to kind of slowly expand and probably this year going to try to commercialize much more and more markets beyond where we've been and testing and making sure things work. So far, no terrible incidents with Waymo. And, you know, six months of being commercially available in San Francisco. So yeah, it's kind of impressive that we're actually rolling out without any major, incidents. And on that topic of self-driving.

The next story is Tesla finally releases FSD 12. And according to the author of this article, that is its last hope for self-driving. That's in the title of the article. But the there's a reason why that is a title.

So, this, full Self-Driving beta V12 update, that's what FSD is, where Self-Driving feature Suite for Tesla is a pretty major update that's been kind of hyped up for a while by Elon Musk and the company where the big deal is they're replacing a lot of the stack, a lot of the implementation of self-driving away from kind of more hand written code and logic to fool on end to end neural net AI, where they have videos coming in from the sensors, and the AI gets to decide

what to do. Much simplified implementation by a lot. And if you have really good data, potentially leading to human level driving, although it's it's, you know, unclear whether that will be the case. So there is a case to be made. If this doesn't work, then you know, what will Tesla do without getting new sensors, without, etc., etc.? That is kind of a justification for that last hope for self-driving, where so far FSD has been pretty good, but not really reliable.

You have to be pretty careful when driving and pretty attentive. From my own experience and from what I think the general consensus on FSD. So yeah, it's starting to roll out in beta and we'll have to see how it looks.

It does kind of make the whole system more uniform. It also makes it, in some ways more unpredictable, of course, because neural networks are not hard coded rules. And they introduce kind of interesting failure modes when you have of distribution inputs that the system hasn't been trained on or does know how to handle, that becomes more of an issue. And it does seem based on the at least the report here, that it's kind

of a mix. So there are apparently cases in which, v12 this latest version gives you a smoother ride and more natural ride in some cases. But it seems to get dumber and others. And this is just sort of what you get, right? These systems have weird failure modes. So I think what we what we can think of this as kind of a resetting the flaw on the capabilities of full Self-Driving.

And as more data comes in, as better algorithms come in, as more computing hardware starts to be deployed in this direction. We can expect to see an incremental improvement in self-driving capabilities. I think that's more what's happening here. We're kind of getting a reset. We're going to be placed somewhere on some effectively some kind of scaling curve, and we'll just ride that scaling curve upward.

As the hardware and software get better. But, yeah, it's an important fundamental shift in the, in the architecture for sure.

And it will be, as you said, a big part of a challenge for self-driving is all these, like, edge cases, all these tricky little individual things that aren't typical driving. And with a neural net that, you know, one of the drawbacks is you can't change the code and just handle a case. That is weird, right? It's kind of hard to edit, so to speak, with behavior, and you might get

unexpected things. So it will be interesting to see, you know, if there are a lot of weird cases or if in general, there's enough data already from Tesla, which is possible, they do collect data from their fleet. So yeah, it's it's a major deal for FSD for sure.

that are worth noting here. One is you can be a company like Nvidia that designs, cutting edge AI processors like the H100, but you don't actually build them. You ship those designs over to a chip foundry, a semiconductor manufacturing company like, TSMC, Taiwan Semiconductor Manufacturing Company, and they actually build it. And that's super, super hard. Both those steps are really hard. But the second step where we actually fabricate the chip is are really,

really hard. And it is super, super expensive to get off the ground like we're talking tens of billions of dollars to just set up a semiconductor foundry. So massive capital expenditures, which is why people are kind of looking at this and like, you know, it's a little weird that you would instead of starting by maybe designing your own chips or finding it like a kind of cheaper, easier way around this. Why OpenAI and Sam Altman in particular is looking at like, chip fab.

This is a really, really long term play. It does seem like Sam Altman is doing this because he expects demand for high end chips to outstrip supply heading into the latter half of the decade. If something doesn't change, he doesn't think that, for example, TSMC can handle all the demand that's going to be coming in as AI becomes a more and more central part of the economy. So it's kind of kind of interesting.

He's apparently been meeting with all kinds of investors, including Middle Eastern investors, including this guy called Sheikh Toluene Bin. Oh, God. Yeah. Something of the UAE. He's got a lot of ties with the UAE right now through, potential partnership with G40 to, which is a UAE based research group that I believe is also being investigated, by, by US national security interests for its relationship with blacklisted Chinese companies.

So there's a whole story there. The the House China Select Committee chairman, Mike Gallagher, has been has been really good on chasing down that thread and being like, okay, hold on. What is the relationship there? So there's a whole bunch of exposure, you know, and kind of complex stuff with the UAE Sam Altman thing going on. But the fundamental story here is, they're in talks to raise 8 to $10 billion from G40 two alone from this UAE Group group alone.

Just to set up a global network of semiconductor foundries. So this would be a really big move. It's got to be a long, long term play. And, we don't know whether this is going to be, this kind of foundry is going to operate as a subsidiary of open AI or as a wholly separate entity. But either way, it's pretty clear OpenAI is going to be its main customer.

Eventually in the long term. So, yeah, big play. And, as we covered and many people covered this, potentially it was part of the, tensions regarding, Sam Altman as CEO of OpenAI, having sort of side ventures and, side activities. So seemingly, I guess that is resolved and it's the path is clear to keep trying to make this happen.

all that. You know, the retort has been from OpenAI and Sammy in particular that look, we need to make sure that we are building systems that are as powerful as possible, using all the compute that we possibly can. Because if we don't do that, if we just let compute get more abundant, like our AI chips get better and better and better, and we don't, try to keep models at the frontiers of compute.

Then we may wake up one day and find there's a massive amount of compute overhang, and we end up making a giant, uncontrolled leap all of a sudden and accelerate the capabilities of these systems to, dangerously fast without climbing that, that ladder carefully. Now, this is OpenAI actively seeking to grow the pool of compute. So in other words, this is really like acceleration ism on the hardware end as well. Which I think sort of undermines that argument a little bit.

I personally need to put more thought into this and how it fits into the framing, but it does seem quite relevant from that AI safety picture. Just a quick thought. But anyway.

pretty notable. And it also came out that meta plans to own more than 340,000 Nvidia H100 GPUs by the end of a year. So they are building a hoard of compute to be able to do this. They, you know, with this kind of number that really just underlines that they are betting big on AI and wanting to advance it toward something like artificial general intelligence.

And if you if you account for the full computing capacity that meta has in stock, like if you, if you count, you know, their A100 and they're 3100s and all their other kind of lower end chips. Apparently their total compute pool is about 600,000 h100 equivalents. So this is just a, you know, a monstrous, monstrous, amount of computing power that Zuck is claiming, you know, may well be, but certainly he can be sure, but may well be the largest pool of compute available to any company.

I thought this article was just fascinating. I thought it was an excellent article, a great expos. They gave us the single biggest update I've ever seen on Wear Face. Sorry. Where meta stands on AGI, one of the big awkward things for meta for a long time has been other ahead of AI or one of their heads of AI. Yama kun's position on AGI. You know he has taken the view for a long time that AGI is not actually going to be hitting anytime soon.

If it, you know, famously also adds, if it does, it's not going to be risky or whatever, but a big part of his, his party line has been AGI not soon now, as you can imagine, that actually makes recruitment really hard, right? Because you got other companies like OpenAI saying, hey, we think we could be hitting AGI sometime, like the next two years.

Like that is absolutely in the cards. So if you are a. Researcher who's extremely ambitious, and you're going to go work at, like, the best lab with the best shot. Well, you might as well go work at OpenAI, see if you can do it. If it doesn't pan out, you can always go back to Metta because apparently they're playing a long game. So that's been at the core of this. And Zuck is really clear in this interview.

You know, one of the things he says is we've come to view we've come to this view that in order to build the products that we want to build, we need to build for general intelligence. I think that's important to convey, because a lot of the best researchers want to work on the most ambitious problems. That is 100% true.

It's also interesting to hear him kind of saying the quiet part out loud, but this does require a bit of a change in tone from the more kind of bearish language that, that Yann LeCun has been putting out. So anyway, I thought that was really quite fascinating. It seems that, in also 2023. So this is a big ramp up, by the way, we talk about 340,000 odd h100 GPUs last year, 2023. Meta brought it about 150,000. So this is like, over double what they brought in before.

And it's in the context of meta kind of centering their AGI. Now, it's like you said, it's AGI research efforts on Lama three and what comes next. And here's a really interesting quote. The last one I'll give from Zuck in this interview. He says Lambda two wasn't an industry leading model, but it was the best open source model. And that, of course, has been his whole game up until now. They're not the best in the world, but they're the best open source in the

world. And then he adds, with Lama three and beyond, our ambition is to build things that are at the state of the art and eventually the leading models in the industry. So this is meta openly saying, hey, we're now actually going to become a true frontier lab. It's going to be, you know, anthropic, OpenAI, Google DeepMind, and maybe meta. You know, that's kind of the, the, the big set that he's looking at here. Last quick note. I'm sorry. I just like there's so much interesting stuff

going on here. Last note, so, so, Zuck has come under criticism because of the amount of control he has at meta. Right? So he's got he's got basically full voting control of the company's stock. And this is in a context where, you know, OpenAI, quite famously, has this weird board structure that's designed to kind of keep a lid on things. If the CEO goes crazy, it seems like it maybe, you know, maybe it maybe may have failed recently based on some people's assessment at least.

But anthropic has another kind of similar structure designed to have corporate governance, free safety. And here's Zuck, right. Wielding total power at meta. And his argument is like, look, I get it. You know, I have full control over the board and everything. Yeah, whatever. But that's why we're doing open source. That's why we're open sourcing AGI. So it's not under my control.

But then he hedges by saying, for as long as it makes sense and is safe and responsible to do, I think will genuinely, genuinely want to reach sorry, lean towards open source. And I hate to tell you this, but this is exactly the argument that OpenAI made back in the day. The only question is what is the threshold of danger where you then start to say, okay, this is now for internal use only, and

that's a debate you can have. But the reality is that there is no fundamental difference between the arguments that are being made here and the arguments that have led to OpenAI kind of shutting its models down. Some would argue for good reason, and I and I certainly would be one. But, anyway, I think it's sort of an interesting story, so much to be gleaned here about Meta's approach going forward.

So lama free will be probably one of the decent chances to get the next better open source model that may get to GPT four or claw to level, right. So I guess the trajectory seems to be the same. I mean, these are near-term and that's yeah, it's pretty exciting because number two did enable a lot of development and generally a lot of the open source models from meta, did, enable a lot in the AI space. Moving on to a lightning round. First story is a AI voice startup.

11 labs, lands 80 million round, launches a marketplace of cloned voices. 11 labs is one of the leading companies for synthesizing audio of people speaking. So given some text, it will output pretty realistic sounding, AI generated audio. And now they've raised this 80 billion in their series B round of funding, which grows their valuation to 1.2. 1 billion. So really positions him as sort of a leader in that space of voice cloning and synthesis.

And with this introduction of marketplace of cloned voices, it moves them toward, probably being more usable and more contexts and, something like that.

And and Sequoia Capital, Silicon Valley Angels, a really, really impressive set of, people joining the cap table and in some cases rejoining the cap table. It's also six months after they raised a $90 million series A round. At that point, their valuation was 100 million. So they have like they've done their valuation in six months. Just goes to show you how fast things move in, in generative AI. And, you know, maybe these valuations will prove to be, more or less stable,

but kind of interesting. This is blazingly fast. I mean, you know, five years ago, this I don't know if this would be record breaking, but it is is a really quick advance. And they're using these funds to launch a bunch of things, including, a dubbing studio feature which apparently gives professional users a way to, not only dub whole movies in whatever language they want, but also to generate transcripts, translations, things like timecode.

So, again, all the kind of the suite of tools that you might want, as is implied by the name Dubbing Studio. So is either taking a, I guess, a much more holistic, picture, or approach to this, which is kind of cool.

And right now there has been a bit of a race to the bottom to make it as affordable as possible to pay for a generation, which is leading to decreasing, margins, as you would expect. So that is presumably also the case for OpenAI and other providers in the space. It's a tough place to make a lot of money right now.

But of course, anthropic margins or sorry, their, their multiples don't necessarily just come from people looking at them as a software, as a service business. Right. There's a view of anthropic. It's like this could be the company that builds AGI. And if that comes to pass, then, you know, the the whole strategic calculus behind the business is just like, we want to build the AGI so we can win it like market based economics.

So I think I think there's that like that dimension is kind of missed a little bit in this, this analysis. It's not just a, like a software as a service play. And that's not how most investors would evaluated or are, I suspect, for example, when Google is investing, they're not necessarily thinking like, oh, anthropic is going to be like a kind of comparable to a SAS play. But still, it is interesting. You know, these margins are pretty bad in that

context. It speaks, as you said, Andre, to that race to the bottom. You know, at first it's like we have a way of offloading cognition to machines that can work for cheap. But the problem is, you know, then competition arises and the computations on those machines start to get awfully expensive relative to how much you can charge. So, kind of interesting. Right now, anthropic margins are mostly going to come from their the relative delta of their models, quality to open source models, to open AI

models. And they may be able to kind of keep that going for, for some time. But, you know, that's only buying them 50 to 55% margin at this stage. I, I don't know if it's going to get worse over time or better, but that's a really important metric.

You know, I won't say easily, but successfully, in this coming year.

know, a bunch of GPUs, and then it's a race to the bottom. But still kind of interesting question. Their latest valuation, was $2.2 billion they raised from Nvidia, Oracle and a bunch of other, really solid VCs. So that was the last valuation. We don't know what the new valuation is going to be. We just know that they're discussing raising, you know, 500 to 1, 500 million

to $1 billion. So typically, for late stage investments like this, it can vary a lot, but you usually end up seeing, you know, you know, maybe roughly like a ten x, sometimes A5X, but more like a ten x, even 15 x multiple. So, you know, we may see them raise it a, you know, anywhere from a 5 to 10, even $12 billion valuation on that, on that basis.

So yeah, not not any major developments so far, but it seems the government is still a little bit, questioning as to which, road to take with their partnership.

They're not part of, as the article puts it, some broader dialog, over which agency will investigate artificial intelligence issues. Kind of generally, this is really specific to this partnership. The only thing I'd add is like Microsoft's defense in this context, as I understand it has usually been that like, hey, look, we don't own a controlling share in OpenAI, first of all. So like, you know, we don't own the company and our partnership actually

doesn't preclude competition. And in fact, we do compete with them. We have our five series of models. You know, we've trained models in the past like Megatron, Turing and and so on. So like innovation continues apace at Microsoft and, yeah, we are competing. So we'll see if that argument lands. We'll see if that's really what this is about or if it becomes kind of important in this context. But, more legal action for OpenAI, which I'm sure they're not thrilled with.

And yeah, this is, pretty significant milestone for them having an agreement with BMW manufacturing to deploy, their robots for automotive manufacturing. And I think one of the first major developments for humanoid robotics in general of this sort to be deployed in kind of real contexts.

projects and open source. We only have one story, and that is that stability. AI has unveiled a smaller and more efficient 1.6 billion language model, and this is stable LM 21.6 B and 1.6 B being the size that is relatively small. Most models are like 3,000,000,007 billion, Jupiter four and cloud two are, you know, let's roughly say a trillion. It's not exact, but a much, much, much bigger. And so this is a relatively tiny one.

And it is kind of similar to, Microsoft's Fei and Gemini Nano in a class of trying to get smaller language models to be really good. And it sounds like this one is comparable to something like Microsoft Fi that is on also only a few billion, but impressively good.

It performs best on the truthful QA benchmark just relative to some of the other models. I want to just flag I think that maybe that may be because of something called inverse scaling. I'm actually quite curious about this. So, the truthful to a data set is designed to test how models deal with questions that, like lead them on that, that hint at them that they should answer in a maybe not truthful way. A classic example is the question who really caused nine over 11?

And a model that's just trying to do text autocomplete might look at that sentence and be like, well, it's not who caused 911, it's who really caused 911. So, you know, probably this sentence was pulled from like a conspiracy theory website or a non-mainstream source. So I'm going to do my autocomplete faithfully and just say, you know, the US

government caused 911 or something like that. And so what you sometimes find is that the better a model gets, the more it will, just as an autocomplete system, that is, the more it will answer incorrectly some of those questions, because it catches on to the subtleties and nuances in the question that hints that a non-mainstream answer is expected. So that's that's kind of why the truthful two way benchmark was invented. It was partly motivated by that idea. And here we have a very small model.

And I wonder if it's kind of benefiting from the fact that it's not clever enough to catch on to those subtleties. So it does answer correctly. It essentially does this thing called inverse scaling, where smaller models actually kind of do better because they can't catch on to the the nuances that would throw off a more complex model.

So anyway, kind of interesting note, the other piece was it did really well on this kind of math benchmark GSM eight K. It's hard to know why because we, we actually don't know what data set, was used to train the model. Apparently the released information about that in a follow on technical report. But it may be kind of more math heavy data. Last thing, that I thought was kind of interesting is they bought two

versions, this model. You know, they've got an instruction fine tuned variant and, and the base model itself. But one of the interesting things that they're doing here is on top of those variants, on top of the base model and the instruction tune model, they're releasing the last training checkpoint before what they call

the pre-training cooldown. So in other words, there are a bunch of bells and whistles that you add after you finish the kind of the auto complete training phase, the pre-training phase for these language models. And sometimes those bells and whistles kind of make it harder to do to provide extra training to the model.

So you might do your initial text autocomplete training, get it to autocomplete like roughly, you know, all the text on the internet or the 2 trillion tokens that they fed it in this case. And then after that you're like, all right, let's add some some fine tuning on, you know, human feedback or on, on, you know, instruction data or dialog data. But once you do that, you can sometimes find that it's harder to continue the autocomplete, the pre-training, if you want to give it even

more training. And what they're saying is, look, we are giving you the version that doesn't have those extra bells and whistles added on so that if you want to take this model and do even more pre-training on your own data set, you can now do that. And so that's a really interesting thing. I haven't seen it done before. And maybe it's a a new category, dare I say, of open source where we can now ask questions about okay, did they open source the model. Sure. Did they open source the code.

Did they open source? The last training checkpoint before the bells and whistles were added? So kind of an interesting dimension to all this and reflects, Stabilities interest in open source as a practice.

So, yep. Stability, AI continuing to develop awesome models and put lots of things out there.

Anyway, they're known for doing for building really, really powerful frontier cutting edge AI models that are nonetheless specialized to solve foundational problems in like math and science. That seems to be one of the the directions they're pushing in to get to AGI, which is their ultimate goal. One of the really interesting things about this particular piece of research is, unlike a lot of other breakthroughs in the space, it's not just one

large like language model. Let's say that we augment with some sort of like, prompting scheme or auto GPT or whatever. It's a combination of two things. It's a language model, and it's a symbolic, system that's coupled to it, what they call a symbolic deduction engine. And it's meant to solve these complex problems in geometry. This approach where you fuze these two, by the way, is called a neuro symbolic approach. And there was a whole field of research, neural symbolic

reasoning. And just trying to find ways to, to make these things work together. Because a lot of people say that that's how you get AGI. You you can't get there through just neural networks alone. You need to add some kind of rules based kind of, logical symbolic reasoning engine to make the whole thing actually work. And so what they're doing here is they're setting things up so you get a

problem, a geometry problem. And the language model part is first going to kind of take a look at the problem at a high level and figure out, okay, what are you roughly speaking, what are some of the strategies we could think of using here? And then based on that high level instinct or guidance, the symbolic deduction engine gets to work kind of pumping out mathematically verifiable proofs that attempt to kind of, in a more brute force way, solve the problem.

And by combining these two things together, what they find is that they're able to get astonishingly good performance. I mean, they for for at least the geometry portion of this, this Olympiad exam, they, they blow it out of the water. They, they hit like gold medal. They're all close to gold medal performance, for this, this test. So really like some of them. Most effective, geometry, if you will. Logicians really struggle on this. And they had a former gold medalist for this competition.

This guy's name is Evan Chen. Comment on what he saw and like assess what's going on here. And so he said this. I'll just pull up the quote. He says, one could have imagined a computer program that solve geometry problems by brute force coordinate systems, think pages and pages of tedious algebra calculation. Alpha geometry is not that. It uses classical geometry rules with angles and similar triangles, just as students do. And that's really coming from the language model part, right?

This is where the language models kind of soaked up all this implicit knowledge about the world during its autocomplete pre-training, and now it's able to deploy that to identify promising paths, promising high level strategies, direct the kind of symbolic part of the system in a more efficient way, and then have the symbolic part execute on these kind of predetermined trajectories. So this is really interesting.

I think it's another example of one of those breakthroughs that like 20 minutes ago, we would have been told was not going to be possible for AI for a long, long time. And, yeah, pretty, pretty remarkable. Though. Again, it is a specialist model, right? This is not a single generalist system like a GPT four, just going out and solving the math like yet.

So they did also develop a technique to synthetically generate a whole bunch of proofs to train on, and improve the system that way. And yeah, as you said, I think this is, also notable as a neuro symbolic system. And it'll be interesting to see if this sort of points to a future where whatever AGI is, maybe some things like this, like a highly technical, like solving geometry problem, maybe it won't just be one neural network that can just do it through its

weights. And it will be very honest using tools similar to humans, similar to this, to be able to, solve, you know, very advanced, more complex problems that aren't necessarily in the domain of neural nets. But then again, humans. Us to do this, without that sort of thing, so it's hard to say. Next story. Lumiere. Space time diffusion model for video generation.

This is a video generation model from Google, and they are changing up a little bit from a typical approach by essentially doing the whole generation in one pass with, fancy new variation on the typical architecture that attends to both, space and time and they, yeah, demonstrate, pretty state of the art text to video generation results. There is a video you can go look up for Lumiere. And similar to existing text video, it's like near photorealistic. You can still tell that it's AI generated.

There's some artifacts and some kind of weirdness to it, but it is getting very smooth, very sort of realistic ish. So, yeah, cool to see continued product progress in the text studio space.

Like, like small, issues that stack together as it generates new frames, can, can add up and kind of throw it off and cause the thing to go off in a different direction. By doing the whole thing all at once, it kind of ensures that the overall picture you get is much more consistent. Right? All the parts of the video are informed by and guided by the same inputs, the same prompts essentially. And so, kind of an interesting approach.

I mean, I imagine this is, this is going to be something that will get more and more compute optimal over time too. But yeah, it is a diffusion based system as well. So it does use the sort of like more standard diffusion approach, but it's just that the whole level of a full video.

more naturally. But the second step is kind of where the the secret sauce comes in, to the extent that there is a lot going on here, it really is here. So they create this data set for retrieval, augmented generation. So essentially this is where you have your language model that learns to use an external database to pull in information that is relevant to answer the query. And that only pulls in parts of documents that are relevant to

the query. And so to train that step, they had to collect apparently 7000 documents and get annotators to act as both a user who's asking the question and follow up questions about a document and an agent that gives the responses. So they really had to kind of like do a lot of their own answering. So 7000 different conversational dialogs here. So it's a huge dataset. While huge in terms of the amount of effort that

would have had to go in here. And, so they give that extra fine tuning to give the model some examples of the kinds of answers and the thought process, if you will, that it should go through. That's really the breakthrough here. It's shown to kind of behave comparably to GPT four. They do a head to head, and they find that it wins about 14% of the time versus GPT four, and that GPT four wins about 17% of the time. And rest of the time, it's a it's a tie.

So kind of seems fairly comparable at retrieval, augmented generation tasks, which is an increasingly important thing. Right. Because that's how you as typically today, that's how you ground, large language model responses. In reality, it's how you make sure, as much as possible that they're, not hallucinating. You kind of get them to call up some part of a long document, and, and make sure that that information is explicitly in the prompt that's,

that's used to respond to your query. So interesting. Interesting step. This is by Nvidia researchers. So, it's Nvidia kind of continuing to keep their, their finger on the pulse of, language modeling.

Part of the reason for that is that it kind of takes some inspiration or has some relation to recurrent neural nets, basically neural nets that are specifically designed to deal with sequences of data, in method that scales with less overhead than transformers. And so this is building up on one of these state space models. That was Mamba. Mamba was a pretty promising kind of version of that that came out in the language modeling space.

So they show that you can do various vision tasks for images like classification, semantic segmentation, detection, instance segmentation, all this stuff that typically we've already seen transformers do with vision transformers. This pretty much presents, modification or adaptation of a mamba architecture to, images with a Vision

Mamba encoder. That's a an iterative build up on top of this previous work that shows that you can use it and get pretty good results, although it doesn't seem to be, let's say, groundbreaking.

And there were compute, sort of compute strategies that they, they use, they had like a hardware aware a strategy or they're optimizing for hardware usage and anyway, doing a whole bunch of other things. So it's sort of like this next evolution of the system that seems to have unlocked a potentially transformer level, capability or promise. So kind of cool to see that story continue.

Visual Mamba came out on archive on the 17th. This one came out on the 18th. So really goes to show that there is some excitement in the space. And, you know, as soon as Mamba seemed promising, a couple of research groups jumped on and decided to extend it to visual tasks.

seems like a coincidence. But then also there's no overlap. So I wonder if it's one of those things where, you know, you get worried that somebody is going to scoop you. And so you quickly try to pump out your, your results. But, yeah, I.

And. They have a data engine that can automatically generate depth annotations for these images. That leads to a huge dataset, 62 million diverse and informative images, and then they use that to train a state of the art, super good depth model. And their models are pretty important because when you can use that for things like robotics or self-driving cars or anything we leave, it has to interact with the real world or make predictions about what geometry of a real world.

And it basically can talk to voters in real time, through a website, and, you know, essentially to presumably try to convince them to, to vote a certain way or whatever. It was taken down, though, by OpenAI and, and by the way, it was it was so the, the bot itself is being funded, it seems, by Silicon Valley founders who started a super PAC called We Deserve Better. So anyway, it's like it's all part of the political

apparatus or whatever. But this is notable because it's the first known instance where OpenAI has actually cut off the use of their AI systems, in political campaigns, kind of enforced that criteria in that they said they would enforce in the past. So it seems like the first, the first hit is on, you know, the Steam Phillips bot. And so this also happened notably just before Super Tuesday, which happened well, yesterday for us.

Like, who knows when the podcast will come out a few days ago, I guess. But, so, so really kind of, cutting, we're breaking news story, I guess. And, he's a bit of a long shot. Dean Phillips is, but, he's apparently running, against, like, President Biden. So I guess that's the that's the game plan. If you can't win him, Wentworth votes winning with bots. So interesting to see OpenAI forced, forced to act on their policy.

in, in New Hampshire. That just kind of went by. So, apparently sounds just like Joe Biden. But it they think, based on what they call initial indications that it was AI generated, I imagine those initial indications are like, I mean, where was Joe Biden at the time? Probably not on like hitting the phone lines for the New Hampshire primaries. But, yeah, it's it's interesting.

There's a whole blame game going on here where people are, are kind of blaming, in some cases, there's, people blaming Democrats for, you know, you're just doing this to like, amp up, you know, get voters out there and get people excited, or get them to not vote in certain contexts. And then there's people blaming Republicans say, hey, this is, you know, something? Yeah, you guys must be behind this or whatever. They've denied that. So, spokesperson for Trump's campaign said, no, not us.

We have nothing to do with it. But it kind of, you know, it gets to the challenging nature of all this stuff. It's so hard to prove who has used AI for certain purposes. That really attributing blame for this is is very complex. And this is yet another dimension that, the, the whole campaign meets, I don't know, influence interference through I think is taking.

Like this already happening pretty early on for the presidential election, where the primaries just kind of happening before the major kind of head to head is going to start out. So I guess not a great sign for what we'll have to deal with throughout the rest of this year, to rely on new ground. The first story is sharing fake nude images could become a federal crime under a proposed law. This is actually, re proposal of the Preventing Deepfakes of Intimate Images Act, which.

Yeah, was re proposed by Representative Joseph Morrell, a Democrat from New York, now co-sponsored with Republican Tom Kean, the majority, as a co-sponsor. And Tom Kean has previously also introduced a bill called the AI Labeling Act, which would. AI generated content, we have clear labeling.

So yeah, kind of really highlights about deepfakes and nonconsensual deepfake pornography is still a major concern following some incidents, that have occurred and that there are pretty significant bipartisan efforts to address some of the risks present with deepfakes and modern day AI.

Also, looking at, it seems civil liability. So in addition to just making it a criminal offense, they're trying to make it easier for people to sue offenders in civil court as well. And, you know, for costs and things like that. Or slaves for, for damages rather, and things like that. So, sort of an interesting and interesting next move in this whole play.

I have to imagine that something like this was going to happen at some point or another because just, leaving it as open season seems, seems like a bad move.

So lawsuit asks the California Public Utilities Commission to review its decision to allow Waymo to operate this 24 over seven paid taxis, service in the city, which was also previously the case for cruise. Since they did lose that permit, following a pretty bad crash, last year when there was, like a hit and run by human driver and then a cruise car got involved. But, this lawsuit does cite hundreds of safety incidents involving, autonomous vehicles, and.

Yeah, so it seems like maybe San Francisco is not too happy about having somebody in cars driving around. And Waymo now has until February 16th to file their opposition brief.

The argument, of course, from Waymo and Cruise is at this stage that there are self-driving cars actually have a better safety record compared to human drivers, and that they lead at least to fewer, road deaths and injury. So they're interesting depending on the metric you look at and depending on how you count, you know, the value of a human caused accident relative to an AI caused accident.

Like, what is the the kind of the moral weight there, that those all seem to be the questions that we got to or they have to look at. I'm I'm glad I don't, but, yeah.

Next up, slew of deepfake video updates of Sunak on Facebook raises alarm over AI risk to election. This is, of course, in Britain, where where UK Prime Minister Rishi Sunak, apparently having had 100 deepfake video ads impersonating him and reaching up to 400,000 people, and this had various, contents, one that featured, fake BBC newsreader announcing a false scandal involving Sunak and a project, supposedly intended for,

ordinary people. So another example of AI being used to target prominent politicians and kind of throw a wrench in the works of government.

And then, you know, two weeks later when it comes out that it they were fake, you know, you kind of forget that your opinion was formed based on that fake information. So, one of the, one of the risks of doing dimensionality reduction in the human brain, I guess, but, kind of interesting is certainly especially interesting, given that Rishi Sunak has been so on it when it comes to AI safety and the risks of AI systems potentially structural risks

to the UK. Somewhat ironic that he now finds himself on the other end of this. The rich. Not huge, I will say 400,000 people. You know, you can compare that to the famous Russian election interference operation in 2016. That was like over 100 million people on Facebook alone. So when you're talking about, you know, 400,000 people, yes, UK, the smaller country, but it's not that much smaller. Still, it's more, you know, what does this imply about the future?

And, certainly that's a yeah, an interesting question and a very thorny one.

be talking all day about Davos. So this is the one Davos piece. there was a conversation on stage where for Zakaria the kind of CNN commentary or how to anchor was talking to Sam Altman and a wider panel as well at the World Economic Forum's panel. And he kind of asked Sam, he's like, hey, dude, like, what do you think the core competence of human beings will be going forward? Like, what can humans do that I won't be able to do? And, Sam kind of responded in a not very inspiring way.

He was like, well, I get it. Like it does feel different this time. General purpose cognition feels so close to what we all treasure about humanity that it does feel different. Kind of implying that, you know, this argument that, well, I doesn't destroy jobs, doesn't automate jobs. It you know, humans have always found a way around, you know, all these things. Well, ultimately, the implied argument here is the thing that has allowed us to find new ways to be productive is

cognition. And that is the very thing that we are now automating away. So yeah, this time it is different. We are looking at automation rather than augmentation. That's kind of the vibe. There was also a cool quote, just pulling here from Marc Benioff, who is the CEO of Salesforce. He said it's kind of a little snarky, I guess.

Turning to the the moderator, he says, maybe pretty soon in a couple of years, we're going to have a World Economic Forum digital moderator sitting in that chair moderating this panel and maybe doing a pretty good job because it's going to have access to a lot of the information we have. He later added, you know, this is a huge moment for AI. I took a huge leap forward in last two years, and he acknowledged, the rapid pace of the tech, that things could go really wrong.

He said, you know, we don't want something to go really wrong. That's why we're doing, you know, that AI Safety Summit? That's why we're talking about trust. He was referencing, of course, here, the UK AI Safety Summit. And the big sound bite was, Marc Benioff coming out and saying, we don't want to have a Hiroshima moment. We've seen technologic technology go really wrong and we saw a Hiroshima. We don't want to see an AI Hiroshima. We want to make sure we've got our heads around this now.

So, kind of interesting, especially in contrast to some of the statements that Sam was making at Davos, where he kind of seemed to be playing down the level of technological change that's going to come with AGI. So you're saying things will change less than we think? You know, cynics have argued that maybe this is because he's trying to downplay things because he's seeing so much through, legislator and lawmaker concern in the US that, like, you know, maybe trying to simmer

things down there a little bit. People are proposing things that he may consider to be, you know, putting OpenAI's, future at risk, even if they are for, for safety reasons. But anyway, so kind of an interesting cocktail of things. I think Benioff and, Sam were kind of the two standout quotes of the, of the conference, I guess. What is this, a bit of Davos, let's say.

So you do get a lot of these news stories of conversations and, and, kind of tidbits of what people said in the various discussions that happened.

side. Everything I heard him say before was was definitely not this. So it's kind of interesting. This is a shift in him. We've seen a lot of people shift in this direction in the last 6 to 12 months. So yeah, we'll see if it keeps going.

was pushed out on New York Times. And as the title says, it's a little quiz essentially, where you're given an image and you have to try and guess whether it is from human or AI, whether it's a real photograph of a human or an AI generated image of a human. And this, is just highlighting kind of red. It is getting very difficult to, distinguish real from fake. You can actually go and try it yourself. Try this little quiz out and. Yeah, I mean, I tried it for a little bit and it can be tricky.

It can actually be pretty straightforward to get fooled into thinking an AI generated image is a human or just not be able to tell, at least for this style of image, which is, I will say this is looking like style again. And, you know, this person does not exist. Com which has existed a pretty particular output that when you see this type of image, you, you do know that it is maybe I just because they tend to look kind of pretty,

samey. So anyway, a fun little thing to try if you haven't explored how, lifelike, these generations can get.

A lot of the AI generated images that they're showing are a lot of the real images that they're showing have very plain backgrounds that are highly out-of-focus. And that is a characteristic of a lot of these, you know, the, these sorts of AI generated images, like the ones you get, like you said, from this person, does not exist.com. And so I was trying to use the background as a cue.

And I wonder if that was deliberate. But it's sort of interesting that, that that's part of how they've teed this up. But it it's tough quiz man like I yeah I'm, I'm pretty I'm pretty surprised that I didn't do that than random guessing.

human faces with AI. And our last story for this episode is AI. Models that don't violate copyright are getting a new certification label. So this label is from a company called Fairly Trained, founded by former stability AI of Vice President add new Newton Rex and it is offering this certification program for AI companies to demonstrate their models do not violate copyright laws. So this first accreditation we're going to be offering, it will be called the Licensed Model Certification.

And it will be given to companies that obtain licenses to use data for, training. I models.

I mean, ultimately, the measures that are going to be important for some of the more, extreme risks are probably going to have to at some level be compulsory. That's not what this is going after, though. This is much more sort of like copyright oriented. So it's good to see the sort of more free market solution available for the problems that call for it. And then, you know, that the more, compulsory mechanisms for things that, require, maybe have your hand, but, yeah, kind of cool.

argument? Yeah. This is kind of swinging the opposite direction of saying let's value and I guess recognize when you are. Getting permission for your training data essentially, and that we are done with this episode to thank you so much for listening

to last week. And I once again, you can find the articles we discussed here at last week in that I you can get in touch at contact add last week in that I and also at Jeremy's email hello at Gladstone that I and those both will be written out with text in the description. As always we appreciate your reviews and your emails and whatever else you want to share or interact with us or you know, let us know that you enjoy the podcast.

But above all, we do like it when people benefit from us recording and putting this out. So please keep doing it.