UL NO. 454: The First AI Breaches

specifically authorized by your team. Threatlocker helps you do this and provides a full audit of every action allowed or blocked for risk management and compliance. Onboarding and operation is fully supported by their US based support team. Stop the exploitation of trusted applications within your organization to keep you running efficiently and secure. Protected from ransomware. To learn more about how Threatlocker can mitigate unknown threats and ensure compliance

for your organization, visit threatlocker. Communal. That's Threatlocker communal. Welcome to Unsupervised Learning, a security, I and meaning focused podcast that looks at how best to thrive as humans in a post AI world. It combines original ideas, analysis and mental models to bring not just the news, but why it matters and how to respond. All right, welcome to unsupervised learning. This is Daniel. Okay. The first AI breaches. What do we have here? Oh. Saw the comet last night. Yeah.

This is really cool. It's been out for a little while. It's actually going to be out for a few more days, and I think it's getting a little less bright, but. And coming out a little bit later each day, I think maybe like 10 or 15 minutes later each day. But my girl took this very, very cool, very, very cool. Uh, she actually spotted it with her, um, naked eyes at first.

She was the first one to see it, and then we all started pointing cameras and binoculars at it, but she actually captured this with, um, her iPhone 16 Pro, so not bad at all. Yeah, basically, the iPhone catches a lot more than the naked eye, and she used light or night mode and also long exposure. And yeah, it looks pretty good. Uh, it's an 80 000 year orbit time for this particular comet. And, uh, the head there, which was quite bright, is about two miles across, and

the tail is something like 18 million miles long. And for reference, it's 8 million miles to the sun. So it's. What is that? Almost three times two and a half times. The tail is two and a half times the distance from the Earth to the sun. And the head is two miles wide. And it won't be back for another 80,000 years. She's like, I guess we won't see it again. I'm like, I don't know, eat your greens, gotta Good exercise every day. You never know how long you can live. Okay, so, uh. Yeah.

Wrote a tutorial on how to use any hugging face model with a llama. So I want to open this up real quick. It's a super quick tutorial. I'm actually I think I already recorded the video as well. Yeah, I think I already recorded the video as well, so it should be a video soon. But look at this. It's super simple. You go to the models page on hugging Face. You basically get the URL. You click on this right here, files and versions, which you can see here. You get one of these files like I got the

smaller one, the four gig one. You edit a model file and this is all you have to put in there. Look at this. It's insane. So you basically do um, from is all you need actually is from. Okay. And then you just put the path to the file and then if you want to add a system, um, prompt you can. So that's that's pretty much it. And you don't need a system prompt. Okay. You can just use the user prompt which is what you send into the model.

But then you do this this command here. So so again we're trying to get a llama to run all these models. So once you have this model file it's saved. You just do this little command right here and then this this name that you give it. Lexi writer or Lexi, writer or writer or whatever you want to call it. You just give it a llama create writer. Okay, so you just made its own model. This takes like one

second to complete this little piece right here. And then you run llama run in the name of the model you created. Llama run. Writer. Llama run. Lexi. Writer, whatever it is. And now you're actually talking to that model, that hugging face model that you just downloaded. So this entire process might be like three minutes. Five minutes, right? Depends how long it takes you to download The the file.

So you just went from having like a couple of dozen models available inside of a llama to having tens of thousands or hundreds of thousands. So really, really cool. And my buddy Marcus Hutchins and I disagree about Elon, about a couple of things about his politics. Um, well, we we agree on his politics. I just think Elon's going to come back. I think he's still liberal. I'm hoping he's still liberal. Um, I would say I'm hoping he's liberal. Not a liberal. Not a current liberal. Um,

but I'm hoping he's more balanced. I believe he's more balanced. And he's going to return. Marcus thinks he is Sith. He is turned Sith. He is straight, far right, and he's not coming back. And I want to say, I do agree that that's possible and I'm worried about it, but I'm hoping it's not the case. And I think it's not the case. I think after a bunch of this craziness blows over in the next year or two that he will start showing his his more liberal and

balanced side, that that's my hope. He also believes that Ellen is basically I don't want to put too many words in his mouth, but I think he believes that he's not a true innovator. He got lucky. A lot of his stuff is garbage. Um, it's just like there's there's no way any of this is going to go well for him. Tesla is going to be basically bankrupt within a year or two, I believe, he said. And I've got a link to the whole conversation we had. And we were we were texting on the side as

well during this. Um, and then basically I'm like, hey, look, should we bet on this? Um, and so he offered a couple of bets and I said, I need these to be measurable. So I offered three measurable bets. One, the Tesla stock would hit 250. Um, you know, it would hit that point by June 30th. So middle of next year it would hit 250. End of next year it will have hit $300 at least once. It would have got to that point. Um, and I actually think it's going to be higher, but I just wanted to

have it 250 and 300. And then that Elon would, in a liberal way, in a principled way that defends something around anti-authoritarianism, something freedom oriented. In other words, Trump would be on the wrong side of this and Elon would be on the right side of it, the correct side of it. And I argued that that would happen by the end of next year. Okay. So that's three different bets, each for $1,000. And uh, he accepted. So

we got three bets out there. Kind of reminds me of this thing that I've been obsessed with lately, which are these prediction markets. Which one is manifold? I forget the name of the other one starts with a P, but, um, yeah, pretty excited about these. I've barely started messing around with them, but I feel like I want to use them more. Okay. I did a talk for a UN group called Ypo, which is a intellectual property, and it went really well.

And I want to thank Olivia Fabrizi, I believe, is how you pronounce her last name for being not just a great host, but actually, really, she has some really great questions. She seems very interested in all this AI stuff, very knowledgeable, and she was kind enough to host us in this. And yeah, it was great. It was great conversation. I listened back to the talk which which I hadn't done before, and it seemed like it went pretty well and good questions at the end. So I encourage you

to check that out. Thanks to Nudge Security for sponsoring and let's get into security. Okay, so an attacker accessed moi. I don't know how you pronounce that. Moi, moi. I feel like you could have done a better job with the name, but, um. AI chatbot database. Okay, so this is I've been talking about this for a long time. A lot of people who are anti I have been talking about this, but I am obviously pro AI. But I absolutely believe this is going to happen. It already

has happened and it's going to get worse. This AI chatbot database is full of like basically AI girlfriends is is the vibe that I'm getting. In fact, let's let's go pull it up. What are we doing? This is why we have video. All right. Select your login. Don't think so. Okay. Yeah. Notice any, um, notice any trends here? How many guys do we have here? Is this a guy? I think the bottom left. We got a guy. Top right or top left? We have a guy. Uh, I'm not sure on this one. And then the rest are

all girls looking a little young. Honestly, uh, if I'm being frank here, but anyway. Um. Avatars. Okay, so basically, it's avatars. It's Companions. This whole scene is getting really big an eye for lots of reasons. There's companionship, but obviously, I mean, this whole girlfriend sex like fantasy angle is is so obvious and it's so obvious that that's where a lot of attention was going to go, both from

the user side but also on the builder side. And so this company, their database got breached sensitive user interactions. It had the chat history okay. So the user sign up you look look at this. You sign up with you look look it's got federated logins okay. These are your actual user accounts okay. So people sign up with their user accounts. Like if I click on this okay I'm going to click on it I'm signing up with my real stuff okay. If I sign up with this.

And then I have this chat with this, this avatar, this AI over there and whatever I say on there. And now that's what gets compromised. Okay. So including sexual fantasies, and the user accounts were linked to people's personal addresses. I mean, it's so obvious that this was going to happen and it is now happening. It's going to be it's going to be nasty, right? So input validation issues,

prompt injection, all of that. But the vast majority of damage this is what I'm saying afterwards will come from customers giving their souls essentially to small startups in the AI assistant AI girlfriend space. This doesn't have to be like a sexual fantasy thing. It could be trauma. It could be somebody using it as there's other sites just

like this, but it's more therapy oriented. So it's like you have a best friend to talk to you and you're talking about your favorite show and blah, blah, blah, and like, you develop this relationship with it, but you might have been saying some really nasty stuff about your parents or about whatever, maybe about your friends, right? Maybe you're really sad and you think your friend stabbed you in the back or whatever. So you're talking all this

crap about them and that gets released. I mean, we're talking about intimacy here. It doesn't have to be of a sexual, you know, sort of flavor. Right. So we're talking about the the closest form of intimacy, which is what you have with like this friend, you you can tell anything and you can't tell anyone else. A lot of people, they don't have any friends. So this is like their only friend. So you give it your trauma, your political opinions. Yeah. Fantasies, whatever. And they totally get you.

But this, it might be like a nine person startup. They have no security. Right. I mean, startups get hacked so easily. It's so easy to hack one of these little startups. They have no security, they don't know how to do security. And once they do, I mean, think about the extortion campaigns here, right? Assuming the people have money, which a lot some of them will and some of them won't. But but I mean, you could just be like, hey, here's a clip from this thing you were talking with

your avatar or whatever. You shared this trauma that you had. You shared this talking crap about your friend. Um, I mean, there's just so much possibility here for malicious use by an attacker who gets hold of one of these databases. So something to think about, Castillo says. A ransomware attack led to a theft of sensitive data. I mean, 200GB. They said it wasn't credit card info, but basically, Castillo got breached. Miter introduced caldera bug bounty hunter plugin horizon three.

Researchers detailed how they identified new vulns in Palo Alto, and they went all the way to full system compromise. And there's patches out for that. Wayback machine got breached six gigabytes of a SQL database, 31 million user records, sites still down, but they're working to get it back up. Researchers from Eset have discovered two sophisticated tool sets used by nation state groups to go after airgapped devices. I see Airgap, I think Israel, but they're saying this is

possibly Russian drop zone. Thanks again for sponsoring. Cybernews says Google's Pixel nine Pro sends data packets to Google every 15 minutes, including location, email and phone number, even with GPS off. And they're using Wi-Fi to to estimate location. UNODC warns that Southeast Asian scammers are using deepfakes to enhance pig pig butchering. Yea, pig butchering these. These are nasty.

It's basically someone builds a relationship with you over a period of time, and basically, especially for older people who are lonely and who have money, obviously. Um, so basically you become their friend, then this person gets in distress and you're like, oh, I can send you money. And

that's kind of how it's done. Usually. Salt typhoon Chinese group exploited backdoors meant for lawful data requests Ukraine has sentenced two hackers linked to Russia's FSB and the Armageddon Group to 15 years for cyber attacks on state institutions in absentia. So they're not there. They don't have these people, but they have called them out and sentenced them regardless. OpenAI has stopped another 20 foreign operations using its stuff

to sway political opinions and meddle in elections. They said they haven't seen any really nasty malware being created because they have a lot of a lot of protections built in for that. So that's that's good news. But still being used for. Yeah, to try to influence campaigns. Basically private intelligence groups like Recorded Future and Flashpoint are changing intelligence by leveraging tons of data from the internet, including

dark web. And I love this. I love this vibe of small companies competing with corporations and small Intel groups competing with, like, you know, big Intel groups. In some ways, study says popular car brands like Hyundai, Kia and Tesla are collecting driver data, including voice recognition and camera footage, and sharing it with third parties. I'm going to look more into this because I'm curious how much Tesla is

sharing because I have a Tesla myself. Pentagon said the US will send a third missile defense system to Israel, along with 100 troops, just to operate it to improve defense against Iran. Okay, so AI tech, if you use ChatGPT, try this prompt just for fun. So this is a meme or whatever that's going around. And essentially what it is is it's saying go into ChatGPT and ask it from all of our interactions together. What is the one thing you could tell me about myself that I may

not know? And the idea is it's going to come back with something and you're going to learn some insights or whatever. And after it gives you an answer, you ask it for another thing. What else may I not know? And then what areas might I be holding myself back? And here's what I got back. You seem to have a constant drive to synthesize ideas across different domains, almost like a mental architect. And as soon as I read, like the first sentence, I'm like, uh, this feels gross.

I don't like it. And so this is what I wrote about it in the newsletter yesterday. Basically, it feels very complimentary and a bit like a horoscope. It feels like a scam, right? Like it's just trying to make me feel good about myself now. Simon. Um, I. I can't remember his last name. Um, I always mess up his last name. But Simon, he talks a lot about AI stuff, and he basically had the same exact observation. And he pointed out, in addition, that it's not actually

doing introspection on all your conversations. Okay, so so what you actually have to do, if you want to do that, you have to go get the conversations and send it in this thing that it built here. I know specifically, I knew immediately when it came back. It's working off of my system prompt. So if you are a paid member and you have anything inside of your your config for ChatGPT, it's basically here's who I am. Here's how I would like you to interact with me. Right? So

I have stuff in there that mentions human 3.0. It mentions like the type of stuff that I'm interested in because I'm using that to have it shape how it responds to me when I ask it a question about whatever I want it to answer me in a specific way. So all of that is what it's basically going off of. And it's essentially hallucination. It's making up all of this

stuff to make you feel good. And in fact, if you go and ask now, um, I think they might have modified the output and basically said that, no, we can't give you deep analysis. We don't actually have the history of all your conversations readily available. You would need to put them in. It's giving some kind of a warning to counteract people doing this meme and thinking the wrong thing about it. But yeah, here's what I wrote about it yesterday. It's like, well, everyone's going to agree.

If you know, something comes back and it's like, you know, I just think you're so insightful and you're so smart and you're trying to help the world and all this, and it's like, oh, wow, that it's so accurate. It's so accurate. It really gets me right. This is the same trick that horoscope and palm reader people use against you is to, like, just basically find special ways and unique ways to compliment you. And it's like, oh, I've never seen how awesome I was before. Thank you. So

I asked people, you know, curious what you get. And I got back a whole bunch of responses and it was basically the exact same thing. And a couple of people responded and said, look, it's just going off the system prompt. It's just hallucinating. So I think we all sort of collectively figured out it was kind of lame. Apple's AI researchers found that llms from meta and OpenAI struggle with basic reasoning, and they've got a whole paper

basically going into that. I think it's more accurate to say that it's easy to disrupt the reasoning than to say it doesn't have any. Geoffrey Hinton, often dubbed godfather of AI, has won the Nobel Prize in Physics for early work in neural networks. Congrats to him. And one thing to notice is that he is in the Doomer camp. And anytime somebody gets a Nobel Prize for crazy work in something like really groundbreaking work, you should listen, especially

if they disagree with you. He disagrees with me, so I'm listening. Musk unveiled Tesla's new robotaxi self-driving electric vehicle without a steering wheel or pedals at the We Robot event, and there was so much hate about this. In the first version that I recorded yesterday, I went into a whole like political thing, which was too much. I don't want to do that again here. So I'm going to

try to overview instead of it. I would say that everyone or most people, most people are very anti like militantly anti Elon or they think he's like the new Jesus and I don't feel like there's very many people who see the subtlety and nuance. And if you go all the way down to the bottom here, I think it's because not enough people are reading biographies, right? Biographies, by definition, are written by or written about people who

have done extraordinary things. And if you go and look at a whole bunch of inventors like Edison and Tesla and people like that, um, they tend Ben Franklin, they tend to be crazy and weird and, like, not take showers and have weird social interactions and, like, sleep under their desk and just smell, like, absolutely horrible and all sorts of things like this. Right? They tend to be

very strange and oftentimes they do bad things, right? And also they're down on themselves and they have these up cycles like manic depressive type vibes, or they have a Jesus complex, or they hate themselves and they think they're not good enough and they're horribly jealous of everyone. Like you're not finding horribly stable people, like very regular people, right, who get biographies written about them. Usually not always the case,

but usually they tend to be kind of weird. And I think that's important when you look at somebody like Elon, because I feel like people are looking at it through the lens of today and what's like considered normal on X or in, you know, in California or specifically in San Francisco, in California, like what is considered acceptable behavior. And I think the question is, if somebody is an extraordinary creator, which we also have to like, agree on that.

I would say he is, but let's say that you agree he's an extraordinary creator and you're like, yeah, but it doesn't matter because he has all these flaws. I ask you again to look backward at all the different things that have been created in the world, and what percentage of those were created by essentially a crazy person, crazy and or broken and or messed up and or in need of therapy and or gross or in some way, you know, counter normal. And I'm not saying excuses, right.

I'm not saying it excuses all the people in the past either. I'm just saying reality tends to look like this. So we should keep that in mind. All right. Deals, deals, sales staff given two days notice to return to office full time. So I've been talking about this quite a bit. Essentially, return to office is a way of doing firing essentially riffs right toe equals riff. Not always, but in many, many cases. It's like it's a cheating way of doing it.

Co-Founder of Robinhood, Baiju is going to collect and transmit solar energy using infrared lasers. And he's basically going to I believe he's going to send the energy back down to earth using a laser as well. So it'd have to be like a receiver. And I'm like, well, you need you would need to be able to target this receiver while in space, moving very fast, right? Going around the planet like every hour and a half. That's very, very fast. But you'd still be able to hit these receivers.

Actually I think they would be geosynchronous. So you wouldn't actually have to they wouldn't be moving in relation to the receivers. Maybe not sure. Either way, you are technically building a space laser, and if you're sending enough energy to be useful on the ground, you're also sending enough energy that could potentially be used as a weapon. So just expect to have at least a questionnaire for the military and or a request for purchase from the military. U.S.

Department of Justice considering breaking up Google after court said they've crushed competition. Google Chrome, Android. Yeah, lots of different products they have potentially. I agree with this. I agree with Scott Galloway that a lot of these services need to be broken up, potentially even Apple, even though I love Apple and I don't think they've really done anything

wrong at some point. It's like a reward. It's basically saying you are too awesome, therefore we have to break you up and not necessarily like an indication that you're a bad person. You know what I mean? Ticketmaster is the first to use Apple's upgraded wallet tickets for iOS 18, giving us stuff like venue, maps, parking, music playlists, weather forecasts, stuff like that, stuff like that. So that's good. Anything

to make Ticketmaster suck less? That's what I wrote. New HBO documentary claims Canadian crypto expert Peter Todd is the mysterious inventor of Bitcoin. Satoshi Todd says nope, I was too busy with school and work at the time, so it couldn't have been me. Which is exactly what Satoshi would say for Taiwanese employees at Foxconn's Zhengzhou plant. World's largest iPhone production facility have been detained by Chinese authorities.

Nobody knows why they assume politics. Humans. Looks like Christopher Columbus was a Sephardic Jew from Western Europe. I think they're saying Spain can't remember if he was Spanish or if he was Portuguese. I can't remember that whole vibe. But they're saying, yeah, Spanish Jew. And I think Sephardic means I can't remember exactly. I think it means like Israeli Jew as opposed to European Jew, which is Ashkenazi. I believe that's the case. JP Morgan and Wells Fargo

dips in profits. So they say it was geopolitical tension. I saw Bank of America saying they had something similar happen. So yeah, banks are sort of struggling. Although I did see Goldman Sachs say they were doing okay, I believe I can't remember the details, but some mixed results on the banks. Your brain changes based on what you did

two weeks ago. So this is basically saying that it's not just what you did yesterday or the day before, but that there's like accumulated downsides of having crappy diet and exercise and sleep over like the course of two weeks. So pretty cool. I mean, essentially, to me this says keep doing your routine because like it accumulates, the benefits accumulate. American Heart Association outlines a strict protocol for taking blood pressure.

They're saying you have to sit calmly with an empty bladder, and that too many people are measuring right after running around or while you have to pee. And basically this this is inflating Leading the hypertension numbers. Boeing is cutting 17,000 jobs, 10% of its workforce. Federal emergency workers in Rutherford County, North Carolina, were temporarily moved after reports of an armed militia threatening government personnel, essentially because they think

they helped somehow create the hurricanes. I assume because they work for the government. Elizabeth Landau says single cell cyanobacteria can anticipate seasonal changes by sensing day length and preparing for winter. Single cell organism can sense seasons. That trips me out and kind of explains how trees start, you know,

turning fall and dropping leaves and all that. United Airlines is adding new routes to lesser known destinations like Bilbao, Faro, Madeira, Sicily and Newark trying to get away from crowded hotspots. Essentially got some journals from Alexei Navalny while he was in prison, for he died in prison. Retail sales have dropped from 7.5 to 7% of employment over the last decade, losing 850,000 positions despite the US adding 19 million jobs overall.

Looks like Wegovy and Zepp bound might have been the cause of US adult obesity rate, dropping by two percentage points from 2020 to 2023. And speaking of that, new GLP one weight loss drugs are coming in pill form soon to basically replace the weekly injections. Dhairya Karwa Mirza, self-taught Kurdish astrophotographer. Look at this image that she took. Extraordinary. Absolutely extraordinary. Look at that. Isn't that cool? Looks kind of oblong. Not sure why. Maybe the shadow. Anyway, very

cool ideas. Gullibility. Not disinformation. So I don't think the US has a misinformation problem. I think it has a gullibility problem. It's not that we're being fed too much crap, it's the fact that we're actually eating it. Some two large number of Republicans now believe that the Democrats are sending hurricanes to Florida because it's election time. That's a population problem, an education problem, not a conspiracy theory problem. So in infosec terms, we need to reduce our vulnerability,

not try to get all the threats removed. Right. Because the threats will always be there. And in fact, the threats are going to get better. The only chance of actually fixing this is education about how the world actually works. And unfortunately, the far left and the far right seem to have like lost touch with that, like the basic facts of reality, because it's not just the far right that's doing this, right. Remember the anti-vax thing that was far left before it was far right? It was far left.

And then Covid happened and then the far right jumped on. And I guess now the far left and the far right are anti-vax. Not really sure, but the point is we have a vulnerability, which is a lack of education and a lack of understanding of how the world works. That's the vulnerability. Threats are things like misinformation. We can have a threat and not be vulnerable, and we still

have lower risk. So that's the vibe there. Discovery swarm, OpenAI's new experimental framework for building and orchestrating multi-agent systems, feels a lot like long chain to me. I didn't see so many benefits. I'm going to mess with it. More command tools. I like really good set of tools here. RFT Delta tldr s-oxide http I fcf. Yeah, lots of different tools here Xvm better vim mode for ssh. I'm

sorry for zsh. Love this thing. It's syntax highlighting. It's basically a full featured version of vim on the command line, so I use vim mode for my zsh have been for years, but this gives you like syntax highlighting. It gives you the ability to use like the surround plugin, all kinds of stuff. Theano 3.0 AI powered API documentation tool. I updated my post on dynamic content generation. I think this is going to be insanely disruptive to so many industries.

Highly recommend you go check that out. Essentially, what it means is people are going to produce raw content. Like for example, this is raw content of a full podcast, right? And this has got me in this bottom right here talking to the camera. And let's say this, this whole episode is like 20 minutes long. Well, let's say that, um, you have a, like a five minute bus ride or a five minute bike ride, or you're waiting in line

for five minutes or whatever. And and this is like, I don't know, three, five, five years in the future. And you're like, hey, I want a summary. What did Daniel put in his podcast yesterday? Your AI is not going to go and find the video and speed it up. In fact, your AI is just going to read, watch, consume the whole video if it hasn't already, and it's going to make a version of it. Maybe it's my face, maybe it's me in a video. Maybe it's only audio

because they can actually look at anything right now. But anyway, it's going to play in your ear a five minute version. It's going to pick the best stuff out of here that you actually care about. How is it going to do that? Because your eye knows you. Your eye knows what parts of these which which of these stories are actually good or most pertinent to you. It's going to take out the garbage. It doesn't like those stories, thinks they're dumb, thinks it was filler or whatever, and it's

going to build a clean little clip. It could be a 32nd clip, but it could be read in the voice of Richard Feynman. It could be read in the voice of Agatha Christie. It could be it could be anyone. It could be Taylor Swift. Right. Telling you the news that that came out of the Unsupervised Learning podcast. Point is, your eye is going to be giving you content in any format, in any presenter deepfaked exactly the way you

want it from 10s to giant series. Okay, the original content, which lives in a textbook somewhere, or in some YouTube video or in some blog post. That original content you never have to mess with again. Your AI is going to consume all of that and then dynamically build the content for their principal. So that's the idea there. Augment UI use AI to prototype front end designs. Software engineer pay heatmap across the US, the digits of pi are

not random. Passbook lets you create an Apple Wallet, pass from any QR code and export it to wallet. How I animate three blue one Brown. It's one of my favorite YouTubers, describes a whole bunch of science stuff and recommendation of the week. If you want to calm your nerves during this next month and a half going into this election, go read about the Civil Rights movement and how disruptive that was to politics in the US. And it was actually, in a lot of ways, way worse

than what we're seeing today. So we survived that really bad stuff. We probably will again survive this election cycle, probably. And the aphorism of the week, what is to give light must endure burning. What is to give light must endure burning Viktor Frankl. Unsupervised learning is produced and edited by Daniel Miessler on a Neumann U87 AI microphone using Hindenburg.

Intro and outro music is by zombie with a Y. And to get the text and links from this episode, sign up for the newsletter version of the show at Daniel miessler.com/newsletter. We'll see you next time.