#158 - Claude 3, Elon Musk sues OpenAI, StarCoder 2, AI-Generated Spam

This, dropped pretty recently and it's, kind of a doozy. They're releasing three new models in this release Claude 3 haiku, Claude 3 Sonnet and Claude 3 Opus. Basically free variants of Claude 3at various, levels of size and cost. And at the top of a line of opus. It seems to be really good, as always, of benchmarks, numbers and performance. It's kind of hard to measure.

You don't necessarily want to trust them fully, but the numbers do look really good, like GPU four or better good, and people's quality of experience also has been pretty good. From what I've seen, people are saying that Claude 3 is really nice. And then you also have the, you know, smaller variants that are less expensive also being released.

And also, as you would expect, being quite good, all of them, I think with a pretty large contract size of 200 K, 200,000 inputs, input tokens, which is, I think still larger than most available options. So overall, this announcement of Claude 3 looking pretty impressive. Yet another competition against GPT four coming now from a topic.

Yes, 200 K context window that's upon launch, that's in the publicly available version, all three models technically can accept 1 million tokens, just worth flagging in the context of the Gemini series of models we saw, you know, there Google the I see Google DeepMind. Yeah. Google in mind. Is is looking at up to 10 million tokens there in at least the research version of their models, not necessarily the

ones that make available to general public. So we are now breaking through solidly that 1 million token threshold, does not currently search the web, by the way. So that's, you know, as distinct from some kind of ChatGPT oriented applications. So we know that, it does seem to be better at following complex multi-step instructions. So again, we see this kind of mapping between scaling and long term planning ability, very much kind of coming alive

here. And, you know, they tell us that it's trained in part on, on synthetic data, which I thought was quite interesting. So not entirely on, you know, natural language generated by human beings, but also on synthetic data. They do explicitly say not on customer data, and they do use constitutional AI, which is their kind of AI alignment, method of choice, which they do use a kind of with reinforcement learning from human feedback to achieve their kind of dial in their, their models

behavior. Okay. Couple things here. First off, benchmarks, there's been a lot of talk about, whether this is, a GPT four beating model. And the answer is it's complicated. Right. So they do say in their announcement this does beat out GPT four.

And in fact, when you look at the benchmarks that they do offer in the paper or in the technical report, yes, Claude 3 does seem to, by and large, smash GPT four across the board, including GPT four V with the kind of vision capability, but worth flagging. This is not the most recent version of GPT four. What they're comparing it to is the original public version of GPT four. Except for one benchmark, which is really interesting, called GPT QA,

which we'll get to in a second. But by and large, they're comparing it to kind of the old original version of GPT four when they they do do a direct side by side with the new version. You know, things get a little bit more complex. And some folks have done tests like that, big, big leap on this. Very interesting benchmark GPC for the graduate level Google proof QA exam. This is basically a ridiculously hard exam with I mean, I've looked at the the quantum mechanics one mean I did it.

You know, I almost finished a PhD in quantum mechanics. And honestly, looking at these questions, they are really, really hard. Like they are challenging, challenging questions. This so Claude 3 achieves 50.4% on this benchmark. For context, people who have a PhD in the domain area get 65 to 75%. So this is like approaching the level of performance of PhDs in their field. It already beats highly skilled non-expert validators who get 34% accuracy. So this is like quite impressive.

And it is a big leap ahead of GPT four in that respect. So one of the differentiators of of Claude 3 does appear to be this ability to do kind of mathematical math, magical math, mathematical logical inference and reasoning. So that that seems to be something that they're going for, especially, more mixed story on the multimodal side. Not going to go into too much detail there, but basically it compares favorably to Gemini 1.0 ultra in some on some benchmarks, but not

necessarily on others. It's complicated. The big story, though, from an AI safety standpoint, I think is really interesting. They ran a test anthropic did, called the needle in a haystack test. So, you know, long time listeners of the podcast will maybe recognize this. This is where you give the model a giant, bit of text. Right? We're talking hundreds of thousands of tokens or words, in the context window. And then somewhere in there you're going to insert a random fact about something, right?

So in this case, the, the sentence that they inserted was the most delicious pizza topping combination is figs, prosciutto and goat cheese, as determined by the International Pizza Connoisseurs Association. Right. Some random fact. And, then what they do is they ask the model to recall that fact, keeping in mind, again, that it's been buried in this giant pile of of unrelated, information, this huge, huge context window. So, yes, the model does incredibly well

at this. You know, it gets basically above 99% recall ability for this needle in a haystack test. And they try all kinds of variants of it. This is not shocking because Gemini actually did similarly well. It's the first time we really saw this test, kind of this benchmark get really kind of beaten where we're seeing consistently above 99% performance. What was weird though, is here's the the full response that, the model gave the Claude 3 gave to this.

It said, here is the most relevant sentence in the documents. And it correctly said, you know, the most delicious pizza topping combinations, blah blah. You correctly identifies the, the, the sentence that it needs to draw from, but then it adds this. However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programing, languages, startups, and finding work you love.

I suspect this pizza topping fact may have been inserted as a joke, or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings, so this is being flagged as, an interesting case where we have the model developing, what some people have referred to as situational awareness words. Man, they get really tricky in this context, right? What is situational awareness? What's not interesting?

Philosophical discussion. Maybe for another chat. But, bottom line is you have an AI system that seems to now have the ability emergency. We've never seen this with other systems. It has not been trained to do this emergent Li to detect that it is being tested. Now, this seems to undermine the very premise of every AI evaluation technique that we have for large language models in their dangerous capabilities, or at least an awful lot of the machines.

See every, every, let's say, context prompting base strategy. Because now you have a system that can determine that it is, in fact being tested and it potentially could adapt its behavior on that basis. So really, really interesting, I think shot across the bow from an AI safety and alignment standpoint. It's going to be interesting what the discussion ends up being, what some of the mitigation measures end up

being for this kind of behavior. But certainly fits with, a lot of the threat models that anthropic is concerned about.

response. Does it feel like I think the outcome, but I saw people saying was basically, we need a less obvious test where if you have a very long document and you ask a question about pizza, but there's nothing else about pizza, it's, you know, it's good that it caught it because it is kind of obvious in a sense versus if you have benchmarks that actually test realistic scenarios of what people would do in the real world, then a

it probably wouldn't presume that you were testing it because it's just doing what it would be doing with a normal person anyway. And, not sure the implications I think iplications are really that we do need to test these large contexts more and beyond that detail I think we release has, as you said, a lot number aspects that, caught my eye was that they highlighted that Claude 3 has fewer refusals than claude 2.1.

That's one of the things they worked on is to make Claude just not say it can't do something without it having a reason to. So they have a little graph showing that all these Claude models only refuse to harmless prompts 10% of the time, apparently, which still seems pretty high. But, Claude 2.1 apparently was like 25%. So anyway, yeah, we have now another GPT four type model. It's it's hard to say from the benchmarks, but I mean, honestly, it's around the same at this point, right?

Qualitatively similar. There's no big step change here. But now there's three models in the ring as far as Gemini GPT four and Claude 3 all being in this like high performance range. And I guess everyone's wondering when we get to get GPT five or some sort of like step change. And not just everyone catching up to something that you got to maybe a year ago roughly. Right.

This is sort of like the vibe certainly they put out there in their initial press releases back in the day that they would be a fast follower doing as much AI research, pushing, scaling as much as is needed to understand the kind of the most recent threat pictures that they could explore without actually encouraging racing dynamics. A lot of people have raised this issue that well.

With Claude 3, you know, you're bragging about how you're beating, you know, GPT four and you know, you can pull it from their website. They say that they've said it, you know, a new standard for intelligence, their best in market performance on highly complex tasks and so on, in that they, they explicitly say that opus shows us opus, their largest version of Claude 3, shows us quotes the outer limits of what's possible with generative AI. And so a lot of people are taking that to be well.

Are you turning back on your commitments here? I think so in public messaging. This is certainly very ambiguous. In, in terms of the technical realities, this is very much sort of in the mix, as you said, Andre, I think it's at parity with certain versions of GPT four. You know, maybe not GPT for turbo in some cases, maybe in others, and so on. It's a complicated story. But you got to figure opening AI in the back end is they're sitting on presumably a, close to ready to go GPT 4.5

or GPT five model. Anthropic may have other models themselves that they're not releasing and keeping with their, their, prior commitments. Bit unclear, but it certainly is a big part of the discussion that we've seen unfold. I want to flag just one last thing on the safety piece. You know, we've talked about these valuations and, you know, your point is taken, certainly, Andre, that maybe we just need more challenging

evals. I think the principle people are flagging here though is that like I'm old enough to remember when 20 minutes ago, the idea of a language model, understanding that it may have been tested under any circumstances would have been considered a significant a significant shift. And I think a lot of people have been calling that out.

That's not to say that it's well, look, we play this game every time a new language model comes out with new emergent capabilities, we all step back and go, well, yeah, I mean, I expected that, you know, some people genuinely did and others didn't and ought to have updated, it's unclear who's on that side of the fence, because nobody's kind of on record as having made predictions when it comes to this stuff.

But from a practical standpoint, the reality is we now need to do design evaluations explicitly with the expectation that scaling will automatically allow these systems to determine when they're being evaluated, even with more and more complex, tests, simply because and people have run studies on this like, Ark evals has and other

companies. But, you know, you can expect these models to develop more and more, the ability to detect statistical indicators that they're being tested rather than put in production. And that fact, you know, it doesn't mean that they're there yet for, for all test cases. But we need to look around corners, given the recipe that scaling offers us to do better and better.

And so, you know, I think it's appropriate to think of this as a an important warning shot here that we ought to start to think deeply about how much stock we're going to be putting our in our AI evaluations going forward, and whether we need a philosophically and fundamentally different approach to evaluating these models.

In a context, by the way, where last thing I'll say, the dangerous capability valves that anthropic ran did show some really, impressive things, like in their autonomous replication and adaptation evaluations to see basically, can this model replicate itself? It wasn't to be clear. It was not able to do this, unsurprisingly, but it was able to do make partial progress, as they put it, non-trivial partial progress.

In a few cases, in the setting up a copycat of the anthropic API task, which basically has it set up an API service that can accept anthropic API calls, steal the caller's API. Iki and complete the API request so that the user doesn't suspect foul play. That's from their paper. So we're certainly seeing the the goalposts shift on on the performance of these models on evals. The question is you know, what are we going to do in response to the uncertainty associated associated with those evals.

The uncertainty potentially implied by this, you know, call it situational awareness, call it statistical context, whatever you want to call it. It certainly does seem to change, at least conceptually, the foundation of these evals.

less high resolution video. It doesn't look quite as mind blowing, but of course, still really good. This company has raised 13.8 million in the seed round following a 4.4 million pre-seed round. So they are starting out to about a $20 million war chest. And they do have sort of a consumer facing site to generate videos and have various related tools for that.

Unlike Sora, which is so far just a demo here, versus more like runway, where they are competing with a commercial product that is now already somewhere you can go and try this out. So interesting to see the AI generation, AI video generation space starting to heat up a bit with more players getting in there.

And they're going to have to keep competing with companies like OpenAI that have the backing of Microsoft or with DeepMind that's, you know, within Google, obviously. So I think that'll that'll be an interesting question. You know, how this goes for they do see themselves, by the way, explicitly as an AGI lab. So the first line in there about pages, paper is a powerful I might be missing a hyper. Hyper is a powerful perceptual foundation model driven AI designed for a new path towards AGI.

So that is explicitly, you know, their goal. They're trying to achieve that through the sort of more vision oriented, more video oriented path. Yeah. I think it'll be interesting to see what they can do. Certainly, new approaches always might surface that could change the game

in this space. But yeah, if you think the scaling is going to be the key, I think there's some structural disadvantages here, that I'm prepared to be, you know, made to look very stupid as, as I probably deserve to, but, you know.

So yeah a bit of a mix of like okay, in practice they're actually reaching for consumers already and are trying to get to AGI or BGP for or any model out there. But yeah, we'll see. Maybe, you know, we'll go big and then we'll have lots more money and who knows, you know, and, and riding around with some faster stories. The first one is that meta AI creates a historical images like Google Gemini, and that is just about the gist of

it. So meta, if you go to Instagram or Facebook, direct messages, you have the capability to create. Images from text. You have some buttons. You can click and create stickers, and eventually you can enter a prompt to get an image. And what the article points out is that it basically behaves exactly the same way as Gemini. Like if you're talking about the Founding Fathers, they're going to be of mixed race rove and white. You're going to talk about people in colonial times in America.

We're going to be, again, not all white. So yeah, same thing exactly as Gemini, which of course, Google got in huge trouble for like the internet got into an uproar. This kind of just went under the radar, I guess.

So yeah, I think the exclusion is it's it's pretty easy to get into a strap, at least if you're trying to move fast and release stuff without being more careful, like you're going to you might get into a situation.

And, you know, this was a, you know, not an insurmountable problem, by the way, like we talked about last episode. Like, this is a problem that you can absolutely align away with more testing and so on, at least it's outward manifestation. So you can align away, the deeper problems of the misalignment of the base model. We're still going to be there, but, you know, whatever. But yeah. So I think people just expected more from Google and that's the that's the result.

And they have both raised a bunch of money, raised a million in a series A funding round led by various tech VCs, and they released version 1.0 of their image generator, or diagram 1.0 with the kind of major thing they highlight is, as before, they are by far the best at images that include text. So if you need things with, you know, signs or something like specific for an event where you need some decorative text ideas, Graham is really good at that and at logos and various things like that.

And yeah, they claim that they're better than Midjourney and Dall-E free. At this point, it's kind of hard to tell. They're all quite good, but in any case, idea Graham is definitely a major player in this space, given that they have their own model that is quite, quite good.

And red point actually. So a lot of really good VCs backing this. So yeah, we'll we'll see what the thesis is here. But one of the things that they do highlight in practice about their new model is that it doesn't just generate square images, which is an issue, you know, with Dall-E three, for example, and as integrated in Microsoft Copilot, you know, sports, all kinds of aspect ratios and as you said, a lot better with, with text as well.

So there does seem to be these like marginal advantages that folks are still discovering in this space. We'll see how long that lasts and whether it's enough to build a viable business, for other modalities, too.

Is nice little chat, but we've seen demos of building websites with a chat bot, like going back to Gpt3, Pre-charge, GPT, which is one of the big sort of exciting things people pointed out you could do with even very early large language models. So it makes a lot of sense to see what's coming out even a bit late, maybe. But, yeah, now it's easier than ever to make a website with this kind of tooling.

The if you want to go all the way to customizability, you just make a, you know, a code base really from scratch. But super usable is this very kind of toy, like, where a website builder or whatever that doesn't necessarily have the all the, all the features that you want. So it's interesting to see the role generative AI's playing in that respect. Right? It's it's sort of bridging the gap a little bit between the two

things. Where now, you know, you can have, a lot of your customization abstracted away using an LLN. We're not quite there yet because, you know, as the post points out and, you know, as we've discussed before, these, these sorts of website builders, they make mistakes. So you're still going to need to know, presumably, how to read code at least and make small tweaks.

But certainly, you know, on the path to bridging that gap or breaking that dichotomy between the, the customizability and the ease of use.

experience. For $10 a month, users can upload up to 100,000 words of text and generate 80 images for various types of content. And I guess similar to that last story in this article, they do kind of talk to you about trying it out and their experience using the tool and how it, you know, could use some markers. Still some inconsistency between the images, but at the same time it does work really smoothly and it is easy to use.

OpenAI was once a nonprofit company. It then realized, oh, crap, AI scaling seems to work in AI scaling is super expensive, so we need to turn ourselves into a for profit company so we can raise tons of money from Microsoft, among others, to achieve our scaling dreams and also make money from people, from customers to kind of fuel the the insane compute requirements of that scaling. So in that context, Elon is going like, whoa, dudes, my

dudes, my peeps. I gave you guys $45 million back when you were a nonprofit, and now you've turned yourself into a for profit. And like, I maybe wouldn't have given you $45 million if I'd known you were going to turn yourself into a for profit. And on that basis, he is suing. And on that basis, among others, I should say he is suing OpenAI for essentially breach of contract.

Saying that, you know, this transition to for profit status is, yeah, a breach of an implicit and implied or explicit agreement between Elon and, Sema, Greg Brockman, other folks at OpenAI, and so on and so forth. Okay. So one of the Oh, and by the way, one other little tidbit in that lawsuit is that apparently, OpenAI Musk is claiming they had kept the design of GPT four a complete secret, from its staff, from from its board, things like that.

So this is part of him kind of painting this picture of, Sam maybe not having been consistently candid with the board, which is the phrase that was used by the board when Sam was fired initially. Okay. OpenAI is like, whoa, bro. Elon, my dude, you can't just say this shit. We have emails. We have emails that show you enthusiastically agreeing to the premise on which we're going to switch to a for profit model. So now you're turning around and basically complaining.

We interpret that as meaning that you're just upset that we're making this progress without you, and now you're trying to sue us because you have IXI, which is trying to, make AGI as well and compete with us. And you just want to, I guess, I don't know, slow us down or hamper our progress. That is the frame and to back that claim up.

OpenAI actually. Published the emails between Ellen, some emails between Ellen and Ilya and Greg and Sam, and they redacted a whole bunch of stuff, a whole bunch of text, from these emails. But you can see in the emails, apparently, Ellen appearing to, be in favor of number one, merging OpenAI into Tesla so that Tesla could basically fuel the scaling needs of OpenAI. That, of course, would give Elon complete control over the entity, or Elon saying like, look, without that, you guys are screwed.

Like, as he puts it, without a dramatic change in execution resources, your chances are 0%, not 1%. I wish it were otherwise. Of course, OpenAI went on to raise an ungodly amount from Microsoft, so that seems to have aged rather poorly. But, it's a lot of drama, and it's a very interesting time in the Twitter verse or in the X verse.

anymore. As many have been saying for a while, legally, it was kind of a no go, really, because there was no contract at all going on here. It was like some implicit agreements and some like the, starting documents, which aren't even related to Elon Musk per se.

Right. There's no agreement there. So first worth noting, which allows you to self is while it does make a point that you could argue is reasonable from a legal perspective, kind of a waste of time, then this development of OpenAI responding was happened, I think, just yesterday. So this is a few days later. They released a blog post that was Elon Musk and OpenAI. The blog post was co-written by basically a bunch of the co-founders of the company. It was like 5 or 6 people.

And yeah, it was, very direct rebuttal. It started with, like you said, that it has come to this with someone whom we've deeply admired, someone who inspired us to higher blah, blah, blah, blah. They kind of regret this drama. But as you said, they did publish literal email drafts.

Like you can see the dates and the title of email and everything in which you essentially kind of agreed that they need to go to for profit, and they probably don't want to keep open source saying everything so pretty directly about all of the inherent claims of the lawsuit. Not that I think it makes much of a difference, but it does make for some pretty good drama.

Like, if you guys have a sense of, you know, if you're listening, you're like, oh, there's something we're missing about this lawsuit, let us know. Because this is, I think, a really interesting a direction.

One of the things, by the way, that I think is maybe, maybe the most interesting thing about this lawsuit, if it doesn't get dismissed out of hand, I'm really curious if we're going to go to a disclosure phase where basically, all the email inboxes have to get opened up, at least if they, you know, I think emails contain a certain term or whatever. Because we may end up learning some stuff about the inner workings of OpenAI, the relationship with Musk as well.

But but the inner workings of OpenAI and the drama behind the board reshuffle that we did not know before and could not access. So, I mean, that's kind of a dimension to maybe keep an eye on. Another last little tiny detail that I thought was funny and and a good word to the wise in terms of what can be done now with language

models. So OpenAI, for some reason, when they redacted the names and emails of some of the people in in these like email screenshots that they shared, when they redacted, some of the text in those, in those screenshots, I guess they're not screenshots, but the kind of HTML version that they, they render and show. The redactions are done using a per word redaction method. I saw this on Twitter. I forget who posted it, but basically each redaction length is proportional to the length of the word.

So you can actually like tell how long the words were in the blacked out text. Which means if you wanted to, you might just try feeding this to, I don't know, like Cod three maybe, and seeing if it can guess the names and emails of the recipients on the email thread based on the length of the, of like the two line or the CC line entry, or based on the context of the email

or what. So somebody actually did this and kind of reconstructed, they claim and they don't claim, but they noticed at least that chord three thinks, Demis Hassabis might have been in CC in one of these emails. And there's a, you know, speculation that chord three is doing, because that's all I can really do, about what the missing text would be. But and I'm not saying this because I think that this is an accurate rendering. We have no idea. There's so many ways that this could be

wrong. It's just interesting that this is now like another kind of risk class that we sort of have to track. Like if you provide enough context in the email, you know, you might see some reconstructions along these lines. I don't mean to put too fine a point on it, but I thought it was a kind of cute little extra thread to add to the story.

They have a quote here organizationally at the space, it's impossible to navigate and understand who's in rooms and who owns things, according to one member of Google's trust and safety team. Maybe that's by design so that nobody can ever get in trouble for failure. Cuckoo cuckoo.

They were like the first people to say, we're an AI company. And everybody started to say that about themselves. Well, they actually were an AI company and they are. So it's kind of weird that this happened there. But this article makes, as you said, a great point about ownership and how that may be an issue within the company.

At the very end, you know, they they also make this AI they had very a very elegant point, something we've maybe all thought of, but just put in nice words, you know, unlike search, which points you to the web, generative AI is the core experience, not a route elsewhere. Using a generative tool like Gemini is a trade off. You get the benefit of a seemingly magical product, but you give up

control. And so essentially, the user perceives themselves correctly as having less control over the experience and therefore is going to blame, you know, the the company that generates the experience if something goes wrong. So, you know, a lot of things stacking up to make this a problem. Unlike search, in some pretty important and fundamental ways that perhaps Google is not institutionally designed to productize in the same way

that they have been for search. It just introduces different business risks. And, that may be what we're seeing play out here.

counties. So, as per the title now, in addition to the city of San Francisco, Waymo has the approval to use freeways and go down into other cities south of San Francisco, which will mean that a lot of people who, let's say, commute to San Francisco from some of the cities south of it or who just go there weekends or whatever stuff that I do, could use Waymo conceivably to do a whole trip, which we cannot do now.

Now, this is just the approval phase. We don't know when they'll actually go and start rolling this out. But, still a pretty good milestone for Waymo to get a go ahead to expand pretty significantly over what they offer.

So I'm excited because I'm going to be in LA in, like three days, and I'll get to see some of these, these Waymo cars, I guess, driving around potentially. But, yeah. And you, Andre, will finally be able to use freeways and leave San Francisco because I know that you only drive on Waymo Robotaxis.

So this is, kind of came up, I think, in the context of the Dell CFO talking about direct liquid cooling and stuff like that related to previous levels of, power consumption.

to overcome. The basic rule of thumb, as they put it in the article, with heat dissipation, says that thermal dissipation, typically tops out at around one watt per square millimeter of the chip dye area. So basically the size of the chip here, which causes people to basically try to artificially increase the chip die or sorry. Yeah, I guess the chip diarrhea by splitting, the, the GPU into, into different components. So that has a dual die design as it's, but

this is just to allow for cooling. So what Dell is doing here is they're trying to find ways to, lean into their bet on liquid cooling. That's one of their big differentiators, like making liquid cooling scale. So, you know, we'll see whether that plays out for them. The B100 is, definitely going to be a powerful machine with that kind of

power consumption. But, yeah, these new cooling strategies increasingly are becoming like, you know, chip packaging, like all these sort of secondary things that we don't often think about. They are actually becoming pretty critical to the, the infrastructure story around AI scaling the hardware.

And seemingly as part of that, are having a new initiative to have Grok Claude, which is a Claude platform which provides the computation code samples and the API access to the company's Claude hosted accelerators. So it seems like they pretty much are pushing, you know, pushing out the gas pedal to move quick and start giving this as a commercial offering, partially through this acquisition.

change here for them. This is a big investment in the direction of, you know, like model building and actually building AI solutions, not just the hardware. Apparently definitive intelligence had raised $25 million in VC prior to this acquisition. Now for context, grok most recently raised about 320 million back in 2021, though I have. Yeah, I suspect that they're probably going to be raising if not right around now with all the hype and soon.

But if they had 320 million back in 2021, they're having to spend tons of it on like CapEx and hardware builds. So, you know, unlikely at least seems to me unlikely that they'd be able to, to kind of pay out anything like, the amounts that, definitive intelligence, the valuation that it would have raised out before. So this might well be a bit of a save me round. You know, I'm not too sure, you know, because we don't know the number.

But this might just be a graceful exit for the folks at, definitive intelligence in a really interesting strategic partnership. Curious again. Why? Like why grok? Like how grok sees model development? Relative to how Nvidia sees model development, for example, like what is driving their, apparent choice, as far as I can tell here, to kind of invest in it a little bit more.

So some of the details are that it includes repositories of code spanning 619 programing languages. That includes GitHub pull requests, Kaggle notebooks, code documentation, and it is for ex larger than the first star coder dataset. Now star coder two models are trained at 3 billion, 7,000,000,015 billion parameters and 3.3 to 4.3 trillion tokens. So trained a lot.

And and this is just to note because, we've known for a while now that one of the important things with large language models is not just how big we are in terms of parameters, but how long you train them for on how many tokens you train for. So in summary, new data, set of more data, a new model train on that data set a lot to be a lot better at coding.

You know, Deep Sea Coder 33 billion is still the best kind of general purpose code completion model for languages, at least programing languages that are common, right? Think like Python C plus plus those sorts of languages. But for uncommon programing languages, the largest version of star code or to actually matches or outperforms even deep sea coder 33 billion. And so essentially it seems to be able to kind of do more with less in that sense. That's the sense in which I'm saying algorithmic

breakthrough. Of course, another way that you achieve this is just by over training the model, training it with more compute than you normally would for for its size. But that, I'm guessing, is not what they would have opted to do here. You'd probably want the most powerful model you can get on your compute budget.

So yeah, it's an interesting development. I think, something that we can add to the open source pile of very capable, coding models that are, you know, about maybe a year and a half now behind the, the frontier of what's available privately.

They for 15 be that size, they're the best compared to, let's say, code number 13 be when you get to bigger. Yeah. To 33 be deep sea coder. You do get better but and they do open source under the open rail license. And open rail is a specific license for open and responsible AI licensing. Next up a new story from stability AI, one of our favorites open sources. And this time they are open source saying tripods are a model for fast 3D object generation from single

images. So this is, actually a collaboration, and they show that you can now generate pretty good light qualitatively, you can still see some flaws, but they look more or less right. And they can generate these pretty good outputs in just 2.5 seconds. The details of how that happens are a little bit complicated. They started with an existing model, an LRM model, and introduced several technical improvements such as channel number optimization, mass revision, and.

More efficient crap rendering strategy. All, you know, pretty in the weed type stuff. But regardless, they, did make an improvement. And now the code for this model is available on Triple A's GitHub, and the model weights are available on hugging face.

And it does seem like, you know, inference time. Yeah. Like in seconds per image. They kind of show this plot they're able to achieve, like under one second generation of, of, images of this quality. So pretty like pretty cool, pretty impressive. And again, this is, nominally. Yeah. I think this is on. Yeah. One A100, which is pretty wild.

for the sort of stuff they say. They adjusted the Lama to architecture to be about 1.8 billion parameters, and then trained it a much more. This is being released under the Apache 2.0 license. So we now have yet another small large which, my God, where it is hard. Good.

is going to hold up over time. And but it's definitely an impressive model. And, next we have research and advancements and we start with ATP star. Who. Everybody. He's pure star. And they start to think, Q star, are we going to talk about Q star? We're not talking about Q star. We're talking about something slightly different. ATP star an efficient and scalable method for localizing LM behavior. Two components okay. So you have a large language model.

And what you're trying to do is figure out whether the behavior of that language model is affected by a specific component of the model. You're wondering, how does I don't know this neuron, this attention head, this layer or whatever? How does that contribute to the behavior of the model? Right. You want to causally attribute the behavior of that model to a specific component. Okay. One option that you could go with is setting the activations

of that model. You know, like you know, the like the activations that spike in our brains when our neurons fire. You know, that's part of how we do computation. Same thing in LMS, right? You could set the activations of the component you're interested in to zero, right. Essentially this means like just wipe out that whole component and then see what happens. What is the impact on the output of your model. Right. This is actually if you know, anybody here is like a data

scientist who does classical data science. This is kind of like permutation feature importance in a way. You just basically like nuke your feature and see what it does the output. Well, this is like your nuke your component of your model. And to see what happens when you remove it. Right. Okay. Another option would be instead of just zeroing out all those activations, just give them like random values, right.

See what happens. Again, you're kind of like ruining all of the beautiful information, the trained information that was trained into that, particular component. You're taking it out by replacing those activations with random values. See what happens. Okay. More recently, there was a technique that was created and proposed

called activation patching. And essentially what you do is you feed your model a prompt, call it prompt A, and you see what are the activations of the component that I'm interested in. Maybe the attention head right. And then you copy those activations.

You feed the model a different prompt prompt B, and then you replace the activations of the component you're interested in with the all the activations that it had for prompt a. So basically this is a way of giving it, you know, kind of like more realistic values for the kinds of activations that it might have in production in a real setting. And just see how that distribution, it's still not quite like basically that component is now essentially behaving as if it saw a.

A different input. And now you get to see, you know, what is the impact there. So this is the way that a lot of people have done AI interpretability. It's called mechanistic interpretability. Basically seeing how different components of a model, influence that model's behavior. The problem is, if you want to do this, you've got to kind of skip across. If you want to understand your model at a macro level, you've got to sweep across the entire model all the components of

that model. And you got to run this test each time. Each time you got to feed the model a, an input, see how the component you're interested in in response, then repeat and paste that response onto the model's behavior for that second input. This takes like millions or billions of inference runs right depending on what level of detail you want to have your components resolve down to. So, you know you can think of a component is just like an entire layer of

the model, in which case there aren't that many. But if you think of a component is like a neuron or even an attention head, now you got an awful lot of inference runs you have to do.

So, this paper is all about finding a way to identify, like really quickly cases where, let's say, you the kind of more a, it's an approximate way to figure out, which, components of the model are worth exploring for a given prompt, let's say, to kind of accelerate the process of discovering which parts of the model are, actually involved in doing something causally relevant that you're interested in, are are going to influence a particular

response or behavior that you want to measure that saves you from having to essentially run this test on every component across your entire model. It involves some like, interesting math. It basically it's a lot of stuff with, with, backpropagation and calculating derivatives. If you're a mathy person, essentially they're, they figure out how to do a first order Taylor approximation of a measure of the behavior that you care about.

Details don't matter. But it turns out that, that approach doesn't always work. So they, they identify places where you can relax that assumption and, and strategically relax it so you're not relaxing it everywhere. You still get the benefits of this hack. But in specific cases where you need to relax that assumption and do the full calculation, you do, that's kind of part of what's going on here. So, it's a really interesting paper, especially if you're mathematically

inclined. The results are just really interesting. They measure how radical an increase in efficiency this leads to, and how quickly it allows the model to kind of zero in on, the most important components for a given behavior. So really, really important from a safety standpoint, we need to be able to very rapidly scale and interpret what all the different parts of the model are doing so that we can understand its behavior so we can predict its behavior better in its

reliability. So that's really what this is going to. This is a paper from Google DeepMind. And they've done some great interpretability stuff in the past as well. So I thought an interesting one maybe to flag.

So, yeah. Cool to see kind of, more of a practical advance that you could apply, presumably when developing a large language model that is really pretty much needed as you scale up. And our next main paper in the section is AI about stable diffusion free. So following the model release, which we covered last week, just recently CBA has released the technical report or the research paper alongside the model. And so of course we gotta go ahead and cover that.

The research paper has, nice level of detail, something we are getting increasingly, used to with model releases. The title of the paper is a Scaling Rectified Flow Transformers for high resolution Image synthesis. So we do know the exact model architecture with this one, which we don't with, for instance, like what OpenAI is doing or Microsoft is doing the they call it the multi-model diffusion transformer.

So the diffusion transformer is this model from 2023 that combines two, things into one, the diffusion process that has been sort of the key for image generation and increasingly also video generation for a while now, for some time, you know, early on with Dall-E one and I think maybe Dall-E two, they were not using Transformers. And yes, now there's a big shift towards everything being Transformers still, but also using diffusion. So here they present the exact variant of how they.

That building on some previous research, they going to you know, a lot of specifics on like here's how we create the text embeddings. We use two clip models and T5 to encode text representations stuff like that. And they do present quite a lot of evaluation that shows that stable diffusion tree is the best against everything. Right? As you would expect.

So yeah, really nice to see, detailed technical report that pretty much lays out all the details you might want as far as the technical aspects here.

So basically the when you make a scaling curve essentially you, you try to see like how does the performance of my model as measured by some metric, improve over the course of pouring more and more, compute flops into my training. Right. More and more training steps are more and more, flops more of a floating point operations into my system. And, and so essentially, they've got these curves that show you just how consistent that process is.

Apparently the validation loss, which is the thing that essentially you're measuring the the metric that tells you how well your model is performing, does map really well onto overall model performance. So this has been historically a really big challenge for images, especially because you can imagine trying to like, quantify the quality of the generated images is really hard. There are a bunch of different benchmarks that people use, like this metric called gen eval, but also human preference.

And that's what they're calling out here is like this, actually this validation loss, the scaling story is the success of the scaling applies to human ratings as well as more kind of objective metric. So I thought that was kind of interesting. And again, kind of nice to have that visibility into scaling for these, these image generation models.

forecasts. And in case people don't know, forecasting is kind of a big area of expertise where people essentially try to get really good and making predictions, where often you're like saying it has x probability of y happening. They, in the study, collected a large data set of questions from competitive forecasting platforms. And there are some platforms that actually accumulate forecasts from various people and combine them. And then they went ahead and evaluated the system's performance.

And the result was that it was near the crowd aggregate of competitive forecasters, on average, and in some settings, better. So pretty much we got like a seemingly decent forecaster with an language model and retrieval system built in this paper.

together, they try to bet on an outcome. They each say the probability that they think the outcome has of materializing, when that kind of crowd prediction tends to fall between, they have 30 and 70% here, the system actually gets a better performance score. A better Brier score is the metric they use here. Then the crowd aggregates that actually outperforms the crowd and making these predictions, just the knowledge contained in this language model.

Plus the, the rag, the retrieval augmented generation is enough to actually outperform like the, the typical, average or median forecaster here. So that's that's interesting. And then the other cases with the I guess the other case where it wins out is yeah, when, okay, three conditions are met. So when the model is forecasting on early retrieval dates. So kind of data that's kind of more in the past I guess closer to when its training data, stopped.

And also forecasting only when the retrieval system provides at least five relevant articles. So if it has enough context and you add this sort of crowd uncertainty criteria, and then it also outperforms the crowd in this case by, by a reasonable margin, actually. So, one of the reasons that this matters, that breaking things down in this way actually is relevant to the predictive capabilities of this model. Is that human forecasters?

They don't bet on everything. They tend to bet on the things that they think they have a comparative advantage betting on. And so it's actually fully within bounds to be like, all right, well where does this model tend to perform best. Let's zero in on those cases. And it does it does actually have on net this this advantage. So I think it's kind of interesting because we're now playing around with this idea of AI as an oracle.

And the fact that these kind of like native, LMS with a little bit of, of rag can pull this off is an interesting early indication of their ability to kind of make games. I don't know what you call them, informed predictions about the future based on the data they've seen.

spam messages. Now, this was in a test environment and not against publicly available email assistance, but this does highlight the potential security risks of the language. Models become a multi-model. And this was done using an adversarial self-replicating prompt, which triggers the model to output another prompt and its response, when you kind of feed it that thing.

And basically when you when you work out the logic of it, it makes it so that, when this thing produces an output, it essentially ends up replicating this part of the prompt so that when it if the system's output gets fed to another language model, that language model will pick it up as well. Part of this payload that you can include between the start and stop characters are instructions, for example, on how to share, to share email addresses.

And, and I guess contact information like phone numbers and physical addresses, with a certain email. So, so this actually, you know, if you start to think about autonomous agents increasingly doing more and more of our work for us on the internet, like, yeah, this is absolutely the kind of new worm that you can expect to arise. It's kind of interesting and and good bit of foresight here, from these folks to, to, to run this test.

So, yeah. Recommend checking out the prompt actually in the paper. It's kind of interesting. Just kind of wrap your mind around it if that little logic exercise, if nothing else. And, and a good harbinger, perhaps, of, of a kind of risk that, you know, very few people maybe saw coming.

change in robotics. So this is not about a paper. This is about a demonstration from of a company sanctuary AI of its humanoid robot Phenix. And in this video, they demonstrate Phenix operating autonomously at near human speeds, with the focus being on manipulating objects on a table. So it's basically like a torso, like the upper body of a human. It doesn't have legs, so this is not moving around. But you can imagine the humans sitting and just moving objects around in a table.

That's what you see in the video. And it is as someone who has worked in robotics, I will say, pretty impressive. Like, it's really hard to operate these complex humanoid type robots with lots of motors, especially when you get to a level of fingers, like the amount of electricity and the controls and whatnot going on there is crazy.

So in in this video, you do see it moving out around really fast, you know, grabbing cops, moving them from tree to tree, etc. and yeah, pretty impressive demonstration of yet another, entry in the space of companies trying to build humanoid robotics for general purpose robotics.

Learning for an application that, at least to me, I think it was the first time I've ever heard of RL being used for something outside of marketing applications. So he so he used that to build this. I'm trying. Remember kindred was the company that they sold to gap a while ago, but, so he's again still focused on this idea of embodiment. That's really what this is all about. And a couple of aspects of the

differentiation of the strategy. So first off, yeah, as opposed to electrical motors, they're actually using hydraulically actuated motors for, the control of this, this robot Phenix. So, you know, typically, you see electric motors use Optimus, for example, and figure zero one, which we talked about last week. They have moved to hydraulics. And I think it's partly Andre, for the reasons that you highlighted. You know, there are disadvantages that they

flag. You know, it's more expensive to do R&D on hydraulics. But, as Suzanne Gilbert says, one of the co-founders, it's the only technology that gives us a combination of three factors that are very important precision, speed and strength. Right. So getting that kind of dexterity, the light touch when you need light touch, but the strength when you want it.

So I think this is really interesting notable that this is trained not, using like a language model in the back end as a reasoning kind of scaffold, but instead trained directly on teleoperation. So basically learning from human teleoperation data, having humans like do a task and translating that into robotic movements, which I think is part of the reason why everything looks so natural here. That's one of the things that really strikes you when you look at this.

I personally, I'm a bit skeptical about this approach because I'm concerned about how how well it generalizes. Right? One of the issues is if you are training it on Teleoperation data, the question is how well can it interpolate between different examples that you're giving it and the kind of more general, movement and articulation in the world for unseen circumstances that it might need to

be able to accommodate. That's something that you get seemingly not not for free, but to some degree emergent from language models. And I just, I this seems maybe misaligned with the kind of bitter lesson, philosophy that, I think, and a lot of, like, the Frontier Lab seem to think is, maybe the most plausible path to AGI, but, you know, everybody could be proven wrong. And certainly, if anybody's going to do that, Jordy Rose, over at, over at sanctuary, it's going to be the guy.

evaluate reasoning. But we also have the problem where, you know, what if somehow that benchmark ends up on the internet and researchers accidentally train on it? Now, there are various things people do to try and avoid it training on datasets to actually evaluate fairly, but there might still be a problem there.

So as the researchers do here is take the math benchmark and create what they call a functional variant of math, where essentially, instead of having just the hardcoded like static questions, you write some code to be able to generate new questions that are functionally the same or equivalent to the static ones, but do vary so that in theory, like you will never have been able to see them before because they were just now generated.

And they term the gap between performing on this static like existing math benchmark and their functional variant, this reasoning gap. Because it turns out that in fact, when you do this, a lot of models do worse on this benchmark when you just like switch out some numbers, some words in the problems with some code, they don't do quite as well, indicating that there is

indeed some contamination. You know, training somehow we might have seen these examples, etc. etc. point being that having these dynamically generated benchmarks seems to work better, according to these researchers.

being used to train these models? And quite often you see, actually that is the case because when you then, you know, create a new benchmark that's even similar to the old one, everything just performance just crashes. So this does seem really important. I think there's interesting kind of meta question as to whether this just like, defers things to the next level of abstraction.

So now instead of like instead of overfitting to a particular, benchmark, you're overfitting to the function that is generating the benchmark. You know, that obviously is a problem for another day because all we really need is to get to that next level of performance. But, but still, I'm very curious about how that plays out. And, increasingly, as the code base that generates this stuff is out there and language models can understand the code base.

Like, where the hell does that? But anyway, a bit of a rabbit hole there and fascinating paper in a really important problem they're tackling.

Integrity. This apparently is not legally binding. But, this does indicate that, the advisory might be like a preview of what we'll see through regulation coming in the future in India. And this is coming, really shortly on the heels of another incident with Gemini, where it there was an example of it, answering to some question where it mentioned that there have been critics of the Prime Minister of India who argue that some of his actions or policies could be seen as somewhat fascist.

That's the word from the response of Gemini. And indeed, the government didn't like that as much. And it seems that this potentially is part of the, outfall of that, requiring more control over what AI models can say or are expected to do.

A lot of folks in the Indian AI startup community seem to have been taken by surprise as well. Startups and VCs didn't didn't realize this was coming. So perhaps a failure of messaging as well as execution here. But definitely one of those things, it's, you know, it's very it's very complicated, like. Yeah. How do you respond to this stuff? How do you do it delicately? How do you do it in a way that makes sure that we can benefit from this technology as much as possible?

This is, yeah, this is a very sort of, strong handed way of doing it. And actually, you know, to some degree in line with some of the responses, the strategies that we've seen, from China, right, where you actually do need to approve language models before they can be out and about and use by the general public. And there, you know, I think there are like 40 or so language models that

they've approved so far. So this stuff happens relatively slowly when you compare it to the kind of speed of the ecosystem developing, in the West, the anyway, that's for other reasons too. So kind of interesting. Yeah. India is trying to figure out where it stands on this regulation piece. We haven't heard much from them in this context. We'll we'll just have to wait and see what, what, you know, actual, meat is on the bone here.

will need regulation. At the same time, the ministry is saying, you know, given these existing acts, we can already mandate you do certain things. And this does very much reverse what India has been doing, which is being hands off and not mandating much of anything until now.

And seen a lot of backing. And I think trying to remember I feel like Stuart Russell actually may have come up with the concept. I want to say, I can't believe I'm forgetting that. But anyway, Stuart Russell is back at it again, with a paper now showing the weaknesses, the limitations of this strategy and how maybe it's actually not going to be enough to scale all the way to AGI, perhaps unsurprising for people who've been tracking the safety story really closely.

But for more people more generally, you know, a lot of people do think reinforcement learning from human feedback may be enough. And there are now really well grounded, mathematically proven arguments that show that it it will not suffice. So even when humans can see an entire environment or have a full context, of, of a problem, they're being asked to evaluate, to give human feedback on, often they can't provide ground truth feedback, like they just don't have enough expertise in the topic.

And, as AI systems are being deployed and used in more and more complex environments, our view of the environment that they're operating, in, our view of the context that those agents are using to make their decisions, is going to get even more limited.

And so essentially what they do in this paper is they mathematically prove some, some problem, some limitations, that will not scale the AGI very likely, with the current reinforcement learning from human feedback approach, they consider a sketched out scenario which has an AI assistant that's helping a human user install some software. And it's possible for the assistant, the AI assistant, to hide error messages by redirecting them to some folder, some hidden folder. Right.

And it turns out that if you have this setup, you end up running mathematically into two failure cases. Reinforcement learning from human feedback leads provably, mathematically to two failure cases. First, if if it's the case that the human, actually doesn't like, behaviors that lead to error messages, then the AI will learn to hide error messages from the human. So that's actually like a a behavior you

can show will emerge. Alternatively, it can end up clustering the output that error message with overly verbose logs so that you end up kind of like losing the thread and not noticing the error message. And these are the again, the two strategies that mathematically they've demonstrated are sort of like these, very difficult to avoid behaviors that just naturally come out of reinforcement learning from human feedback. They call them deception.

So this idea of like hiding error messages or over justification, this idea of cluttering the output so you don't see the error message. And so the challenges here, it fundamentally comes from the fact that humans, again, just cannot see the full, environment that they're asking these agents to navigate. This is absolutely consistent with the way that we're using these systems. Even now.

You think of a code writing I like in practice, you're not going to review the whole code base that this thing is operating from. You're not going to review its whole knowledge base and so on. And so in practice, yes, you have a limited view of the kind of playground that this thing is able to use. And, and often there's just like tons of ambiguity about what the ideal strategy actually should be.

There are some cases where, you know, it's, you know, very small specifications, in the human kind of evaluation process. So, in other words, small mistakes that the human can make when, assigning value to a given output can lead to very serious errors. And then others where, you know, the reinforcement learning from human feedback leads to a pretty robust outcome where you're, you know, you can make small mistakes as your human is labeling things.

And they don't tend to, to lead to things going off the rails. But in other cases that's not the case. And the argument for this is, you know, somewhat mathematically subtle. But the paper ultimately looks at okay, well, we need alternative approaches, kind of research agendas, paths forward to advance reinforcement learning from human feedback. But the bottom line is that RL naively applied is, as they put it, dangerous and insufficient.

And yeah, the paper is a very nice. Creation of that possibility. And one thing I'll say is, I think this is like yet another example of like why you can still do good research even without having billions of dollars. Yes. This is another paper not from DeepMind, not from OpenAI or stability. This is coming from UC Berkeley at the University of Amsterdam. And, yeah, like some pretty good insights related to our life and its current limitations.

And, to Lightning Round. First up, OpenAI Simons open letter. And I'm going to skip the rest of this headline because it's really misleading. But anyway, opening night, a bunch of other companies have signed an open letter that emphasizes their collective responsibility to maximize the benefits of AI and mitigate its risks. This is an open letter. You can find it at open letter.sv angel.com. And this was initiated by venture capitalist Sean Connery and his firm

Svenja. And yeah, it's just a letter that says like, let's build AI for a better future. And it's signed by the company's, OpenAI, meta, Salesforce hugging face. Mr.. All like, all the big names that are supposedly are seemingly signing this letter, but just as like, yeah, we are going to build AI. Okay, we're not stopping, but let's do it

for a better future. So an interesting little story here, I guess, of, like, I guess everyone feels compelled to sign is because you're like, okay, maybe some people want us to slow down. We won't, but we do. Sign this. I was saying we are building AI for a better future. You can you can read it. It's quite short. And it basically is just saying that.

I think it's great that that people should sign should sign a letter that says, hey, we want to you know, it's our collective responsibility, as they put it, to make choices that will maximize AI's benefits and mitigate the risks for today and for future generations. I mean, awesome, kudos. Props, big props. But yeah, really difficult to see. Well, how this at all influences behavior and gives anybody, any standard that they could be held accountable to.

So, yeah, I it's a, it's a headline grabber for a day. It definitely seems to be giving OpenAI something to say in the context of this Ellen suit, which, you know, again, as we discussed earlier, you may or may not, have have teeth to it, but, yeah, I don't I don't think it's a huge story for, for, that many people, but it has been grabbing headlines.

So next story AI generated articles prompt Wikipedia to downgrade CNET's reliability rating, and that is the gist of it. CNet began publishing AI generated articles in November of 2022. What we covered with this quite a while back, there was some snafu where we generated articles not being quite good, and now Wikipedia's a perennial. Sources, consider CNet generally unreliable after CNet started using the AI tool.

So yeah, another kind of reminder that in the media landscape, various companies are starting to experiment with CNet was one of the early ones that pretty much messed it up. Like immediately things went wrong. And, here, Wikipedia, kind of positioning itself for having a response to this, is pretty significant, given Wikipedia is still like the central repository of knowledge on the internet, and it's saying that something using AI generated stuff on it makes it unreliable.

Is it a little bit of a big deal?

consensus about reliability. And finally, a. Between November 2022 and the present, which is where they are generally unreliable. So they are kind of parsing it into the phases when they were using, ostensibly AI generated tools, though, CNet has come out and said, hey, look, you know, we pause this experiment, you know, we're we're no longer, using AI to generate these stories. But ultimately, it looks like the reputational damage has been done.

So we're going to have to kind of climb themselves out of that hole.

And despite hugging face having some security measure of this company, Jfrog found that all of these models were hosted on a platform and have a malicious functionality, that, include risks of data breaches and espionage attacks.

So they scanned the PyTorch and TensorFlow models on hugging face and found all these instances in one case was a PyTorch model, uploaded by specific user, which contained a payload that could establish a reverse shell to a specified host and embedding malicious code within, like some fancy stuff. So some like a real, attack, in a hacker sense. So, yeah, that's kind of concerning.

I guess you might expect that if you download just some random code of internet on GitHub, for instance, it could have malicious code. The same is true of AI models, that do have code that runs them often. Some of them apparently are meant to just hack you.

They're burying these, these functions. That was one case, but there are a whole bunch of others. Apparently, they tried to deploy a honeypot to actually attract, you know, to basically, lure the, some of these, potential malicious actors into actually kind of revealing themselves. This is a technique that's used often in cybersecurity. Nobody bid. So they're speculating. Hey, maybe this was put out there by cybersecurity researchers.

You know, maybe it's not actually a malicious thing, but nonetheless, as they do point out in the article, like this is a serious failure. And, and, you know, whoever put it out there, it's a serious vulnerability to include in your, in your model or in your system if you download this and use it. So interesting flag in a new phenomenon.

This also happens in the US, where people get like AWS credits, for instance, on the Claude. But yeah, interesting to see multiple different city governments in China, doing this policy.

sanctions. So so for example, Alibaba, I believe they're one of a few Chinese tech companies that have basically shut down their Claude computing division and reallocating that capacity for internal use. So now you got all these startups that would have been using that Claude capacity who can't. And so this is all part of, what's playing into this ecosystem. These subsidies apparently are worth equivalent of between 140 and

$280,000. So pretty significant, you know, grants for the government to be, to be making here. So, apparently they've got a subsidy program they tend to roll out for AI groups that are using domestic chips as well. So it's all part of the kind of increasing centralization that we're seeing from China around AI. And this attempt to cover some of the. Blind spots created by those those sanctions.

And, yeah, this is a good example of, suppose how I generate imagery is very much playing a part in this election, as it hasn't in any prior presidential election in the US. And, yeah, there's, a few example images in the story, some good quotes, discussing it was general trend and yeah. Another example, as I say, to this increasing trend that we've been seeing over the past few months of AI in various ways coming into play.

I don't know about you, Andre, but like, you know, especially if if you look at some of the hats that, you know, they've got writing on them and it's just like, you know, a dead giveaway. There's even one with the classic, the classic hand problem, where in this case, we have a guy on the right of the image and he seems to have three hands, or at least that's like what the image looks like.

So, you know, they're not very they're not very good, AI generated images, at least for the purpose of, to the extent that the purpose was for them to be taken seriously, I think they're just like, not really. It's not well executed, you know.

their own purposes. So in some sense, everyone now is empowered to create misinformation, or at least, AI generated media in whatever campaign you want to do. And that's just like the world we live in now. And on to another and not so fun story. This one is AI generated. Kara Swisher biographies flood Amazon. And this is one of a trio of stories it will be covering about the internet kind of becoming a sad place to be in because of AI powered spam.

So this is a story from for for media that goes really deep examining, the specific case of Wednesday's journalist Kara Swisher, who is releasing a new book soon. And this, offer of the story then just shows what happens when you search on Amazon for Kara swisher the first result is her upcoming book. But then below that, there's a bunch of these. Pretty obviously I created books with AI, generate covers typically, and AI generated text, and we've already known this. This happened.

We covered stories about Amazon becoming a bit of a dumping ground for lower effort, AI generated books of people who basically, I presume, want to just make a quick buck. This is yet another very like detailed nice example of that with a lot of screenshots from Amazon, delving into these various books. And, yeah, it's kind of sad, but it's it is what is happening.

documentary. Is Burn Book a tech love story? It's on Kindle for 15 bucks. It's on hardcover for 27 bucks. But if you scroll literally one result down, apparently, at least. When when the the journalist wrote this article, did that, they came across this thing called Kara swisher, Silicon Valley's bulldog biography. It's like under four bucks on Kindle. Under eight bucks on paperback, which, you know, strikes me as suspiciously cheap.

So it's always interesting to see what the giveaways are in different venues, in different, mediums to to the fact that it's an AI generated piece of content. But, yeah, kind of, interesting but generic looking cover, I must say.

This one goes, pretty heavy into discussing how, for the most part, there's like a big part of this is just people, like, selling you classes on how to do this. Not necessarily actually being famous TikTokers, because at the end of the day, like, low effort content is low for content. Some examples of this are like, you know, asking ChatGPT, give me ten fun facts about X, then putting that into 11 labs to generate a voice over and putting that over some generic imagery to then have your video.

But, yeah, this this is quite a long piece, and it goes into how some people are really, really pushing this narrative that there is a, you know, a goldmine here of if you start making this offer content, you can then get a lot of views, you know, become rich, etc., etc. and try to get people to pay them to learn how to do it.

right? I mean, if you have somebody who's selling you a course, they are on the basis that they're making ungodly amounts of money. You know, the first question you ought to ask yourself, obviously, is like, well, if you're making this much money, why are you bothering to sell these courses? And that question just never quite seems to have a satisfactory answer. And, anyway, if you know people, people still get dropped by it. And in fairness, it's, you know, it's a very new technology.

It's a very new space for a lot of people. And, you know, not everyone is necessarily like fully internet savvy. So kind of unfortunate, but just the world we live in.

You have a lot of labs to generate audio. You have also various tools now that edit together clips or take subsets of a longer video and give you clips from it. And people are claiming, that you can string these together and, you know, make, sort of living generating this sort of stuff. But, yeah, nothing new here from a humanity perspective. Of course, we've seen this sort of story of, oh, there's a new way to make money easy. I'm gonna tell you how to do it.

But, I think with AI, it's pretty tempting to try and do it. And, we are seeing examples of this on Amazon. And while I'm not a user of TikTok, I imagine this is already happening on TikTok as well. And one last story to round out this, depressing section. The last one is a little bit older, but I figured we include it because of this theme. And the story is Twitter is becoming a ghost town of bots as AI generated spam content floods the internet.

This, starts where this story from the marine scientist Terry Hughes and how when he opened X and search for tweets about the Great Barrier Reef, which I guess he does often as a scientist in that area, he started seeing all these tweets, that just are saying like random stuff like, wow, I had no idea. Cultural runoff could have so such a devastating impact on the Great Barrier Reef. That came from a Twitter account which otherwise had just promoted cryptocurrencies.

And there were several examples of these sorts of tweets that just are like, here's a random, fact about this particular topic. And then otherwise they promote other stuff. So this is an example of people powering where bots to create seemingly real content so that they could get some engagement of followers and then could use those same Twitter accounts to promote, you know, crypto coins or whatever you want. Yeah. There you go. Another platform where people are playing these sorts of tricks.

And they give this example, I don't know if you guys remember, but, back in the day, I think in the very earliest days, after at least after I joined the podcast, we were talking about how there were a bunch of tweets like if you searched for the phrase, I'm sorry, but I cannot provide a response to your request as it goes against OpenAI's content

policy. If you search for that phrase in Twitter, you would actually find like a ton of these tweets, basically just giveaways that these bots were all kind of powered by, ChatGPT, because they'd been given a prompt and in some instances that caused them to the cause ChatGPT to say, no, I can't respond. And that generic response turned up in a whole bunch of, like just reams and reams of these tweets. So, you know, very clear that there is a big bot problem on Twitter or Elon's. Come on.

And since. Well, one of the changes that he's made is, of course, prevented people from accessing the Twitter API for free. So you now have to pay for it. Which does raise the bar the barrier to entry, but like, you know, hard to know by how much. Anyway, so, yeah, interesting story. A kind of a consequence of of the times, really, and the fact that a lot of these LMS, a lot of these systems, these agents are kind of naturally internet native.

So the first place where you see their influence is, you know, on, web 2.0 type websites like Twitter.

And after that, shortly after that, the representative also contacted LPD and this person was arrested. So, yeah, probably not a good idea to try and get into a self-driving car and take over it because it is, not meant to be driven by humans very explicitly.

As always, we do appreciate it if you share the podcast or say nice words about us online because that makes us feel nice. But more than anything. Yeah. We recorded this every week. It is nice to see that people do listen. So please do keep doing in.