From Relay, this is Connected episode 537. Today's show is brought to you by OneBlocker. I'm your annual chairman, Federico Vittici, and it's my pleasure to introduce Mr. Stephen Hackett. Hello, Stephen. Hello, Federico. How are you? I am doing fantastic. Thank you. How are you? I'm good. We are without Mike this week, so a little sad. But everything's cool. He just had a conflict. There's no... It's not baby time yet, as far as I know. It's not. As far as we know, it's not.
I talked to him like 10 minutes ago. He didn't say anything. He just had a conflict today. You would think he would tell you. I would hope so. Yeah.
He's like, oh, I've had a four-year-old this whole time. What? A couple of years ago, one of my favorite video game podcasts, TripleClick, Jason Schreier, co-host of TripleClick, and also a reporter at Bloomberg, surprised his two co-hosts with his second baby that was actually born months before and he didn't tell them and it was an incredible moment on the show. Awesome. Yeah, that was wild. But yeah, hopefully Mike will not try to copy that approach.
It's like the time you told us that you had been using a PC for like six months. Well, I wouldn't say that was as dramatic as revealing a whole baby, but, you know. Okay, they're slightly different. Yeah. A little bit different. We got some follow-up. That's what we do here at the top of the show. And many people have written in about their own iCloud woes. This was the bulk of the episode last week talking about my...
My whole iCloud situation with my legacy version of myself and the new iCloud family, and I couldn't do the additional space. So it seems like a lot of people are in that situation. People are adding their legacy accounts to their iCloud families. I will say, you know, a week and a half in, however long it's been, it seems like the dust is all settled. Except on my Mac, in the Mac App Store, I can't update any apps. Okay. I got to deal with that. Sure. Why not? I mean...
It asked me to log into the purchasing account, which I don't want to do. Have you thought that maybe the Mac App Store is so bad there's nothing to update? Well, currently I've got four. Oh, you do see the updates. Okay. To be fair... Two of them are Safari plugins. Okay. And one is an Xcode plugin. So it's a ghost town in there. Look at you being a developer and everything.
I know, right? Xcode plugins. I didn't even know you could get Xcode plugins on the Mac App Store. It's not really a plugin. It's an extension for the simulator. Oh, still. It lets you do fancy stuff. Okay. I'm a developer now. I'm the developer now. Yes. Yeah. So, I mean, lots of people are dealing with this. Hopefully it was helpful if you find yourself in this situation. A couple of things that I didn't mention that some of them I didn't realize at the time that we recorded.
A couple of them I just didn't get to because of time. One is that I lost all my test flights because I was signed in a test flight with that old Apple ID. Because TestFlight follows your purchases ID, apparently. I mean, thankfully, I'm friends with most of the people whose TestFlights I run. But I'm choosing... I'm choosing... to view this as a clean start. I believe you will call it a blessing in disguise, right? Yeah, that's right.
i like you i'm sure uh i had a bunch of stuff in there that i you know oh yeah expired apps uh i mean going way back to like well years and years ago i kind of i kind of want to look now let's see Yeah, see if you can find the oldest thing in your test flight. And the two most important ones for me, I have the login to App Store Connect. So like, oh, I can just add myself back to Widgetsmith, you know, and underscores other apps.
Well, this is actually quite perfect. So the oldest is TV forecast, which is odd because I still use it. Excellent app. Parcel. package tracking app. But I just think these are like build expired and build removed because the developers have new betas. These are like older versions. But the first app that said... tester removed is a DevonThink 2go version 3 they removed me from DevonThink that's a bummer
Yeah. Once you enter Devon, you never leave unless you're you, I guess. Some real gems in here, though. I'm sure. Yeah, I'm sure. So anyways, if I was on your test flight, you'd like me back. Please let me know. I also was rebuilding my music library. Some people had some suggestions on maybe what was causing this or what the deal was. Our friend Zach had mentioned.
They said that, okay, maybe some of these are like pre DRM free purchases that maybe weren't upgrade upgraded. I don't really know what the, what the issue was, but again, like the test light thing. blessing in disguise like let me just rebuild my library i knew that i had music that was not in the store so i have a copy of my old music library on my desktop and i'm just like as i'm like finding things it's like oh yeah let me just add this back to the library um
But a couple of things to note in rebuilding your Apple Music library. And my previous library dates back to when I first started using iTunes in 2002. A lot of stuff in there I hadn't listened to in a long time that I sort of cut loose. But first thing, music encoded for Apple Music, in a lot of cases, sounds way better than what I had in my library.
I had some ripped CDs, and I did iTunes Match at some point, but who knows what the deal was. I've noticed on a couple albums, like, oh, this sounds better. Or maybe it's been remastered at some point, and I had an old version or whatever.
But the thing that hurt me, and Federico, I think you in particular will appreciate this, too many of my favorite albums now have 20th anniversary versions. Oh, yeah. Like... like quite a few and they're you know they're remixed or like death cab has done it with a couple and they've added like demos to the end of it which you know i'm all for and it's like fun to hear alternative versions of songs and stuff but i just noticed as i was like going through
adding things i was like oh oh oh goodness like yeah a lot of this uh a lot of a lot of pain in the past you know uh so test flights Apple Music. Mac App Store. Mac App Store. And anything else? So far, everything else has been pretty smooth. I said last week that some media wouldn't play back. That's all basically been settled. I watched the first couple episodes of Mr. Robot over the weekend that I had purchased.
You know, years ago on my legacy account, they just played just fine. That show is holding up. I mean, I'm only like three episodes in again, but it's really good. Remember, did you watch Mr. Robot or was it Mike? Oh, no, I watched Mr. Robot. Maybe he did too. I feel like we talked about it back in the day. Do you see Mike as a Mr. Robot person? I can't remember. I thought one of you did and the other of you didn't. So maybe he didn't.
Hey, if you miss Mike on this week's episode, here's something that you can send him. Use your favorite generative AI. It can be ChatGPG, it can be Gemini, it can be DeepSeek, which we're going to talk about. Well, it depends on what country you're in. Well, it depends on what country. And ask your favorite AI product to put together a short... description of what role Mike Hurley could play in Mr. Robot. Wow. If Mr. Robot could feature Mike Hurley as a recurring character, what would he do?
that's incredible uh i'm asking gpto for right now what it says okay uh do me i'm just going to read a couple of these to you okay okay uh It's a list. Number one, the underground broadcaster title. Mike could play a podcaster or radio personality who operates a secret channel spreading anti-corporate propaganda. Yes. Helping F society communicate with the masses. Incredible. Okay. Number two, corporate insider turned whistleblower.
Ooh, okay, because he used to work at a bank. I guess so. Given his real-world experience running a media business, Mike Hurley could be a high-ranking E Corp executive in charge of internal communication. who slowly realizes the company's dark secrets and leaks them. And there are a couple of others here. This one's, I think, my favorite after the first one. Tech culture commentator.
In a meta twist, Mike could play a fictionalized version of himself, hosting a tech podcast that dissects the rise of E-Corp and the decline of privacy, inadvertently influencing key players in the show. Okay, well... And then... The final one. A surreal take, Elliot's consciousness voice, where Mike is never physically present, but his voice constantly plays in Elliot's mind as an inner monologue or guide. Fascinating, but why?
Why would Mike's voice be in Elliot's mind? Okay. What kind of role do you think would fit him best? I'm not engaging with you, Chachi Petit. Okay. I like that a lot. We have some other... I don't know what this is. This is follow. Following. Following. I don't know. Follow FM. Yes. We've dropped the FM. Out of our name. We are now.
officially just relay. Uh, there's a link in the show notes to a blog post I wrote over the weekend. Uh, we got our redesign live new logo. You see it this week in the show art, a slightly tweaked show art, new logo. And I'm really, really happy with this. And if you are a member of Relay, which you can do, there's a link in the show notes to get Connected Pro, which is the longer ad-free version of the show.
All memberships come with access to Crossover, which is this feed where we publish members-only episodes. And in February, I'm going to be interviewing JD Davis, the designer who redid Relay's branding. And so that'll be fun. In the Discord, I'll ask for questions in February. But looking forward to that conversation with JD. JD's awesome. He did a great job with this. And I'm really, really happy with it. Nice. Yeah, I think it looks...
Cleaner, modern, fresh. I really like this change. And it looks very good at small sizes and it's still recognizable even at a small size. So yeah, did an excellent job. Yes. Yeah, that was one of the goals was like, how does this look small? Because usually podcast artworks is up on your phone 400 pixels across. So yes, glad to have that, have that out.
I did want to take just a second and kind of talk about the times that we live in. We were going to do this last week, but the three of us felt like we wanted a few more days to kind of put some thoughts together. We know that a lot of people in and beyond our community are hurting, given the state of things in the U.S. and really across a bunch of other countries. And we want to do what we always have done.
provide a place to hang out, talk about the topics we love with people who respect each other. And the thing I'm honestly most proud of is Relay's community. And in that community, there's... a strong shared belief that everyone should be respected and that we should all practice compassion. And that comes out in a thousand different ways in our community. And one thing it means is that there's no room...
for hatred of other people in our community. And unfortunately, feelings like that towards individuals has become more mainstream. And that's troubling to us. And we want to be a place that we can... Again, hang out with our friends, talk about the things we love, and be in an environment where we can be, where we can know we can be safe, whoever we are or wherever we come from. And, you know, we're not turning our shows into anything that they're not already.
We're going to keep doing what we do. But we did want to tell people that, you know, that's where we stand. And we think that a lot of the stuff in the world right now is not okay. And we... Want to use our platform where we can to protect those who are in our community. Yeah. If I may just add some personal context. Please. These are my personal opinions, but I feel like...
using the exact necessary words is important. So I want to start from my personal belief that trans rights are human rights. And I personally find what is happening in the U.S. despicable and dehumanizing. And I cannot even, because I cannot imagine. I personally cannot imagine what it must feel like to feel like other people want to make you and your identity feel... unjustified and and like something that that shouldn't exist i cannot imagine that but i i
I can relate to that feeling of feeling powerless and feeling like there's nothing you can do. And to an extent, I, you, Mike, we... This network are, you know, it's not like connected can change the U.S. government, right? If they only would adopt the Bill of Rickies. If only they would do it.
But there's a couple of things that I would say. Now more than ever, I feel like, and unfortunately these discussions are... trickling down to italy as well you know because we have a government who very much sympathizes with with the American government. So those discussions about human rights, they're happening here as well. But I feel like now more than ever, it's important, if you feel like it, to be organized.
to descend and to seek a safe space in real life and online. And I think it's important to find your people. And to not accept what is going on. And I know that it's a challenge. And I know that I am saying this from a position of privilege myself. Because my identity is not at risk. But I think it's important for people to be together and to feel those things together. And the second thing is that from my position and I think from our position,
It's important to continue to provide the joy. If you find joy in this podcast, I hope you do. And to continue to provide the utility. Every so often we share something useful. That's also my hope.
But I think it's important. Like once a quarter, maximum. Once a quarter, maybe. Yeah, a segment about task managers. But I think it's important to, at the same time, keep doing what we do. Because in any... dark time you still need to find that light that never goes out to to quote a famous song um I think it's important to continue to provide that service for people who want it because like, you know, I think in any difficult period of your life...
you need to have something that brings you a little joy. And so I think it's why now more than ever, we got to keep doing what we do. If I could hug my listeners, I would. And this is one of those moments where I really miss not having a live show and seeing people in person. But yeah, that's all I wanted to say. No, it's well said. I agree 100%. And it is something that I agree with you. It is hard to...
It's hard to put myself in the place where I'm told that who I am shouldn't exist. Yeah. But that doesn't mean that we can't... first of all, empathize with it to the degree that we can, but also provide a place where everyone can feel safe and comfortable and welcome. And that's what we want Relay and Mac Stories. Yes.
to be and look our our discords are the best places on the internet and our communities are incredible yes and um you know we're still gonna joke about ipod socks yeah yeah uh exactly Sound pretty good to me. Yes. So we are literally going to talk about iPods. We are, yeah. For the next topic. Well, not socks. But we are going to talk about iPods and more specifically, Stephen, I wanted to ask you what's your budget for iPods looking like these days? Because...
If you have an opening, you could participate in the Sotheby's auction for the custom iPods from the late Karl Lagerfeld. It's a massive collection of... How many? 500? Like, how many? 300? I'm not sure. 500 iPods. Of all kinds, including a... Like, this... diamond-encrusted iPod and microphone. There's multiple color variations. I didn't even know that Karl Lagerfeld was collecting iPods. So this entire story...
that this collection is now up for sale at an auction is new to me. Were you familiar with the existence of this iPod collection? No, I wasn't. Okay. I mean, I was familiar with the name. you know uh seeing carl's name float around but had no idea and uh that goes deeper than we think it did so we'll get to that in a minute but um right now the uh bejeweled uh, iPod and microphone 500 euro. So, you know, we'll see how high that goes. A little rich for my blood right now, but it's, uh,
This is incredible. And you can go through here and like so many iPods, there's some custom, custom like colorways of the first gen Nano, which Apple just shipped in white and black. There's red, pink, blue, and yellow. And I'm not normally like a yellow fan. Yellow iPod looks good though. But the yellow iPod looks sick. Yes. It's so good. I'll have a link in the show notes to some of these, and it is just incredible.
You were talking about this on Mastodon, and then we learned a lot more. So what was uncovered? Apparently Karl Lagerfeld obviously was very much into iPods. I saw some posts and some old stories saying that he was basically treating iPods as cassette tapes. So each iPod would be loaded with a specific type of music or a specific collection of albums and so forth. Which, I mean, if you are really into music and the storage limitations of the time where, you know...
the storage limitations at the time. I can sort of see that, you know, if you have an unlimited budget and you want to be fashionable and you are a fashion icon, in fact, and you're really into music and you'll be like, you know what? I am going to accumulate. hundreds of iPods, and each of them I will treat as a cassette tape. Cool. But what was interesting is that apparently Lagerfeld had a whole team of people dedicated to managing these iPods.
Because obviously you needed to sync them with iTunes, right? And so the story goes that there was an entire raid of XServ to run this massive iTunes library. and sync hundreds of iPods to this library. Yeah. Have you seen an XServe raid? Are you familiar with this? Never, never seen it in real life. That's because you've never come to my house. I've got one.
But yeah, terabytes of music, which I mean, think about the time frame of the iPod, right? Like that's that's pretty wild. We weren't, you know, doing eight terabyte MacBook Pros at the time. Absolutely incredible. Man, what a move. It's like, oh, this is my iPod team. On one hand, I feel like I shouldn't be allowed to judge because like...
I do keep several video game handhelds for specific types of games. Sure. But on the other hand, I don't have a hundred handhelds. Like I don't even have 20, I think. But, I mean, I'm also not Karl Lagerfeld. So it kind of cancels out, you know. I don't know, though. What a power move to have hundreds of iPods. servers to manage the syncing. Just the entire story is incredible. Yeah. There are also some modified, I just found them, some modified iPod minis.
including an all white and all black model, which look really cool. Are you sure you don't want to try just like even getting one of these? How much are the minis? The minis are $400. one of just one just a random one this used to belong to you could you could you know you could touch it and be like this ipod was also touched by karl lagerfeld and now i am touching it you know
This click wheel was once clicked by Karl Lagerfeld. Yeah. There's a whole collection of third-gen iPods, which we know were the best ones. And those have like stickers on the back with notes, I guess, with maybe what was on them. Wild. Have you ever attended like an auction in real life?
I have not. I've always wanted to. Me too. And I've always wanted to get into a bidding war with somebody, but that only ever happens on eBay. It's one of those things that, you know, I want to do at some point in my life. Yeah. getting into a taxi and be like follow that car but it's a random car and you gotta pretend that it's important you know that sort of that sort of thing but also like attending an auction and getting into a bidding war for like a totally random item
Like it doesn't have to be important and you have to be like really and oddly into it. You know, that'd be, that'd be fun. Like, sir, are you sure? You want to offer 3,000 euros for this hat? And I'll be like, yes, I need the hat. I do. I do want to do that. I do think it'd be fun. But no, I've never even attended one. It's kind of a bummer. Yeah, yeah. Maybe, you know, maybe one day. Maybe one day.
This episode of Connected is brought to you by One Blocker. One Blocker is a premium content blocker for Safari on iOS and the Mac. In addition to blocking obtrusive ads, One Blocker can block trackers, annoying pop-ups, EU cookie notices, comment sections on blogs and YouTube, and so much more. And they're thrilled to introduce the new OneBlocker 6. It has an updated design with an entirely new interface.
to make the app even more intuitive. Plus, you can expect improved blocking and free ad blocking because users can now enable one filter at no cost, including advanced blocking for YouTube. Then upgrade to premium to unlock all the filters and automatic weekly cloud filter updates. I've been using one blocker across my devices for a long time.
One of my favorite things about it is you can create custom rules for domains that may not fit in in OneBlocker 6's other options, but their built-in options are extensive and the new redesign makes it easier to use than ever. One blocker was featured in the Mac App Store by Apple under the best Safari extensions. And it's available for the iPhone, iPad, Mac, and Vision Pro as a native app. You can get premium for just $1.24 per month.
billed at $14.99 a year, or go for a lifetime license. And a single purchase unlocks one blocker across all your Apple devices, so you can share premium with up to five family members. For a limited time, OneBlocker is offering listeners of Connected one month of premium for free. Premium unlocks all features across iOS, macOS, and VisionOS and can be shared with up to five family members.
Go to oneblocker.com slash connected and use the promo code connected. That's oneblocker.com slash connected. The link is in the show notes and use the code connected for one month free. Our thanks to one blocker for the support of the show. All right, Stephen, have you heard the good news of DeepSeek? I have seen the word a lot this week, but you're like, you're wired in. Yeah, yeah. I've been tracking this entire story.
And it's going to be a long segment. And Steven allowed me to do my research and to take my time. So, hey, I think it's going to be an interesting one and a fun one. But if you're not interested, well, that's your... problem not mine skip to the closing it's going to be exciting this week it's going to be exciting because yeah it's going to be an exciting moment for the show uh so
In case you haven't seen it or in case you've seen this name repeated to death and you couldn't be bothered to look into it, DeepSeq is this new large language model, this new chatbot. that is supposedly rivaling in performance OpenAI, which had GPT, Gemini from Google, Cloud by Anthropic, and not just in performance, but also in how it was put together. I've been trying to make sense of what multiple sources are saying. It's very hard to pin down exactly, also because of the language barrier.
And also because everybody's saying different things. I'm going to try my best and sum it up. DeepSeq was put together, more specifically, the underlying large language model is called DeepSeq V3 by this, this is a word that I learned this week, by a quant, which stands for quanti... Quantitative.
hedge fund. It's basically a hedge fund in China that uses sort of like advanced mathematical operations and whatnot to track market movements and that, you know, whatever hedge funds do. Yeah, I think they... They go around trimming people's bushes. Yes. That's what they do. Come on. Yes. Stephen, please let me move on from whatever that was. So this company...
They were using machine learning and large language models to predict market trends and stocks and all that fancy money stuff. But they had assembled a collection of GPUs. And so the... The story goes that they started at some point a couple of years ago as a side project training a large language model called DeepSeek.
that they were going to use in their main business. Then, last year, they started training with... lower number of nvidia gpus than are typically used in america in american ai labs to train large language models they started training version 3 of their deep seek large language model with a reasoning version called DeepSeq R1 And they have published, so I'm getting to why everybody's losing their mind. They have published a white paper, and this model has open weights.
which sort of means open source, but I don't kind of want to get into that. They have published this model and this reasoning model for free. You can use it for free. There's a white paper that you can read, and it goes into the details of how it was put together. Now, for those unaware, a reasoning model... means that there's a version of DeepSeek when you go to the DeepSeek website or you're using the DeepSeek app on your phone you can enable R1 which
shows you it takes a little bit longer to execute but it shows the model the the cot or the chain of thought of the model it shows the model quote-unquote thinking and trying to understand your problems and basically thinking out loud about the query that you just asked Now, most people haven't been exposed to a reasoning model because both Google and OpenAI, I believe they only make it available if you pay for either Gemini Advanced or ChatGPT Plus and Pro.
ChatGPT, which is obviously the most popular in the world, they don't give you a reasoning model because you gotta pay, and most people don't pay for ChatGPT. And with ChatGPT free, you only get... GPT-40. So what's impressive here, according to the theoretical story of DeepSeek, is that the company used a much lower number of GPUs. So for a much, much lower cost to train this high performance large language model. And they did that by...
Basically, using constraints, there's these new export laws in the United States that were... put forth by the Biden administration, I believe, that basically prohibit NVIDIA from exporting too many GPUs to China. through the cluster of GPUs that DeepSeq already had before the laws were enacted, and also apparently, now we're getting into speculation territory, apparently through black market channels.
They were able to... So some people are saying 2,000 GPUs. Some other people are saying it's 5,000. Other people are saying, no, it's 15,000. But apparently it's a much, much, much lower number. than what, again, supposedly OpenAI and Anthropic are using. Like I saw someone reporting that OpenAI is using half a million NVIDIA GPUs in their data centers.
The theory would be like, imagine that for one-tenth of the horsepower and one-tenth of the cost, again, in theory, because it's not like DeepSeek is publishing their finances, but imagine that for one-tenth of the power... and one-tenth of the cost, DeepSeq was able to match and exceed the performance of ChatGPT-01. ChatGPT-01 being the basic version of the reasoning model. Yeah.
How could they do this? Well, I was going to say that there's speculation that maybe they didn't do it all on their own, right? So, yes. So there's also speculation. So this is where we get to the... It's a complicated story, so I'm going to try and break it down piece by piece. What's clever and interesting about DeepSeek is that... DeepSeq is like, it's not like DeepSeq, they invented the transformer model or the idea of a large language model. They built on the foundation.
of something that was invented by American companies. It was invented by Google. I believe the first paper on the transformer model is... Credit to Google engineers for that. And then later came OpenAI, which had GPT as a product based on that. And I believe OpenAI was also first with a reasoning model. So DeepSeek... This is what's fascinating, that DeepSeq optimized the foundation created by American companies. And they did that by applying constraints.
to their engineering teams that were forced to work with less power and less money. And they did that by basically sort of putting a unique spin on how...
Large language models can be trained. I'm going to try and simplify here because it gets boring fast and also because I'm not an engineer. But my understanding is that Typically, there's a lot of supervised training where humans are actually supervising the model and feeding the model the correct answers and the correct process when they're training.
There's supervised training, and then there's reinforcement learning, where basically you are applying a reinforcing technique to say, yes, model, good job. This is the right answer. Basically, DeepSeq only used reinforcement learning to train their larger language model. And apparently, they used DeepSeq itself. to train DeepSeek V3 and to train R1. And this is where we get to what you just mentioned. Apparently, OpenAI is now speculating that DeepSeek...
used training data from ChatGPT to train DeepSeq v3. So basically the news of the day is that OpenAI is mad. because DeepSeq used data from ChatGPT to train their model, which is kind of ironic. Yeah. The company that scraped terabytes and terabytes of data from the open web is now upset that a Chinese startup scraped their data to train their model. There's a... What do they call it? Poetic...
Justice, something like that. Yes, it's kind of beautiful. Anyway, the result of the debut of DeepSeek last week, DeepSeek V3 launched in beta last month. came out officially last week alongside the reasoning version R1, it took the world by storm. By that I mean that it became the most downloaded app on the App Store. that the entire tech and ai industry lost their minds uh because like they were like how can this
Chinese company that's coming out of nowhere. Spoiler, it's not coming out of nowhere. People who are really into AI knew what DeepSeek was up to. It's just the rest of the world realizing now that there's a public product. But in any case... As a result of the debut of DeepSeek, they basically took how many hundreds of billions of dollars off the US market? Yeah, it was wild. Nvidia shares, like Nvidia lost something like $300 billion in a single day. $600 billion. $600? Okay. Yeah.
So, yeah, we're looking at basically like if you consider the NVIDIA stock that was down 17% or something. Google was down. Meta was down. a bunch of other tech companies were down. Only Apple was up 3%, but we'll get to that later. There is a conspiracy theory here somewhere. And if I were into conspiracy theories...
I would kind of believe it, which is like, wouldn't it be fun if you like this entire narrative? Because you could say like, what if this entire narrative about DeepSeek doing this for cheap? and doing this with like one tenth of the power. What if all of it is fake? What if the entire narrative is fake? But this little story that they put together was a tactic by the Chinese government. to put a little dent into the NASDAQ, into the US market.
and wipe off a little, almost a trillion dollars in a single day. Wouldn't that be fun? Now, that's a fun conspiracy theory. I mean, I am definitely not an expert in any of this. Yeah. But, you know, there's also the angle of like, was it a stock market move or like, is it just China doing China things, right? Like, we're going to get to this, but, you know, many...
While some countries in the U.S. Navy are like, hey, don't use this. This is very much wrapped up in those international politics that are above our pay grade. But, you know, there is an element to consider that, you know, what they say should be taken with a grain of salt until it's verified by other people. Just as the same as we should.
verify what OpenAI and Google and these other companies say, right? Like these AI companies make really grand statements and you've got to put them to the test. Yeah. So even if we set the potential politics of it all aside, I think it's interesting to look at DeepSeek and specifically the white paper that the company published, because it does...
It does paint an interesting picture for the future of American companies and the future of large language models and also the future of open source. So it is undeniable that the DeepSeq team used... a series of really creative approaches to build this large language model. For example, they were able to work with one fourth. of the memory consumption, because instead of using 32-bit floating point operations, they used 8-bit floating point operations. So that's like four times less RAM.
when you're running this in your data center, which is fascinating, using reinforcement learning and with some... So basically, like, I was listening to... a podcast about this yesterday. They were convinced that they could do just reinforcement learning without supervised learning. But then they realized that the model was not responding correctly and it was continuing to like mix and match English and Chinese in the same sentence.
for DeepSeek V3, they started from scratch again and they did some, like they did all reinforcement learning with some supervision in the final stages of the process. So even the story that it was all entirely based on reinforcement learning. It's not correct if you read the white paper. So really some interesting approaches that as a result, like there is a product that you can go there. You can go to DeepSeek on the web or you can download an app on your phone.
and you can talk to it and i can tell you because i have been testing all of these things i can tell you that the performance of deep seek it most definitely rivals oh one and the latest Google Gemini version 2 advanced experimental models. The performance is up there. Now, does that performance come from scraping the same... data sources as OpenAI and Google, I'm sure. Was it made possible because they used ChatGPT responses to train DeepSeq? I don't know. But whatever it is, it works.
And I think there's a few interesting takeaways and also some other things to consider, which I'll get to in a minute. First of all, I think it's undeniable. that this large language model and this product was built because of the work that had previously gone into large language models in America.
Once again, it's not like DeepSeek has invented a completely new technology. They have taken an existing technology and said, but what if we optimized the costs? And what if we optimize the performance? What result would come out of that? So they optimized the foundation of something that was created in America. And I think it is undeniably eye-opening for American companies that...
maybe you can achieve. And, you know, people have been keeping an eye on the open source AI scene knew that this was going to happen. If you follow, you know, if you follow blogs and podcasts and YouTube channels about this stuff, you knew that... This was months in the making, this idea of a free and open source model coming out and matching the quality of all one.
It's been like, even the people who saw DeepSeek a couple of months ago, the V3 beta said, oh, when this launches, it's going to be a big one. But it is eye-opening for both OpenAI and Google. to have this kind of competitor. And I think, realistically speaking, what's going to happen here is that OpenAI will have to move up the timeline for the release of ChargeGPT 03.
Sam Altman has already said on X that they will accelerate the timeline of O3. And O3, by the way, is the even more advanced reasoning model that they announced in December. Such terrible names. Yeah, they have terrible names. Terrible names. OpenAI, they also said they will make O3 Mini. which is the smaller model based on O3, they will make it available for free for maybe or at least up to 100 free queries per week.
And I think on the Google side, because these are the two biggest players right now, right? It's in the sort of models that you can pay for. It's OpenAI. which are GPT and Google with Gemini. And I think what Google will need to do, Google will need to make an even bigger deal of their reasoning model, which is Gemini Flash 2.0 thinking.
These names, man. Slightly better than O3, okay? Slightly better. Flash 2 thinking. It's slightly better. But I think Google wouldn't... The whole interaction with Gemini... when you have to pick a model from a drop-down menu is just as bad as OpenAI. And right now, I think both Google and OpenAI will need to clean up their list of models.
and have something simple. Because DeepSeq is showing that you open, you have one model, you tap a button that says, do you want to have the reasoning one or not? And that's it. And so I think both Google and OpenAI will need to simplify and make available to people a thinking model of some kind. And I will keep an eye on Google. I think a lot of people are under the assumption that Google Gemini is as bad as Google Bard used to be. And I can tell you that it's not like that anymore.
especially the, what's it called? The Gemini 2 Advanced. Oh gosh, Stephen, you're going to make fun of this once again. I'm sorry, but look, I'm not the guy in charge. Gemini 2.0 Experimental Advanced version 1206. It's not my fault. It rolls right off the tongue. But that model is really, really good. That version of Gemini is really, really good. So I think both Google and OpenAI are feeling the pressure from DeepSeek.
There's another conversation to be had about the safety practices of DeepSeek and the idea of censorship by the Chinese government and the CCP. sort of looming over DeepSeek. First of all, it seems pretty much clear that the folks at DeepSeek don't have a safety team, don't have any safety practices in place. in terms of how exactly was DeepSeq trained, especially if DeepSeq trained itself via pure reinforcement learning. They haven't published anything in terms of like...
how safe it is. What kind of content was it trained on? They don't have a dedicated safety team or even person that we know of. So that's a big question mark. And also... Try and ask anything about the Chinese government or Chinese history or Taiwan, Taiwan independence or the Tiananmen Square massacre. And you will be able to see... in real time, DeepSeek R1, exposing its thoughts and censoring itself in real time. Yep. You will literally see the chain of thought stop.
Once the model realizes that you're asking about the Chinese government or Chinese history, it'll say, ah, let's talk about something else. Not great. What's interesting, though, is that DeepSync is open source. And you can host it somewhere else. You can download the model and put it on your own server or your own machine. Now, when you run the model yourself...
or when you're using DeepSeq hosted by somebody else. For example, I believe Perplexity, they are hosting DeepSeq R1 in the United States. I have seen other services already implement a DeepSeq option. And once again, you can run it yourself. In that case, it will not be censored. It'll happily tell you about the oppressive Chinese government and the Tiananmen massacre in the past, and it'll tell you about Taiwan, and it'll tell you about anything.
So interesting that obviously DeepSeek, when it's running on the DeepSeek servers in China, it's censored. But when you run it yourself, it's not. And obviously, like... With DeepSeq, most people are not going to download DeepSeq and run it on their computers. Which is why...
For example, the Italian privacy watchdog, which is an entity of the Italian government, has banned DeepSeek from the App Store. So the DeepSeek app is gone from the Italian App Store and they have issued... a warning to DeepSeek and the parent company in China asking for details about privacy collection, user data collection and user data retention.
and about their privacy policies. And, you know, what exactly is DPSIC doing with the data from Italian citizens? What are they collecting? Why? And where are they storing that data? So this kind of story... I think you will see it happen in more and more countries in the next few days. Because for most people, once again, this came out of nowhere. Even though people who are tuned into the AI industry knew that this was going to happen, for people...
Like, I mean, I saw it everywhere on the Italian newspaper, on newspapers, on Italian news websites. It's everywhere. And so this kind of like... governments getting into this and be like hey hold on a second this model this ai from china what's it doing with the data of our citizens i think it'll happen in more and more places i think it will too and yeah
I mean, the upside to this sort of thing is like, if these models can be created and trained on less hardware, that's good. It's good for the environment. It's good for smaller companies wanting to get into it. good for competition i can't help but think that some of the freak out over this at least in the in the u.s is that oh this has been an american company thing and now it's in china and there's some
freak out around that in corners of the US. But I'll say this, that undeniably, the DeepSeek team... they applied some really clever engineering constraints to come up with DeepSeq V3 and DeepSeq R1. That's, I think, objectively undeniable, especially if you read the white paper. And they, for sure...
have documented some fascinating techniques that because of open source, I'm sure others will copy and implement. And that sort of, it's the rising tide that... uh what's it the floats all boats like what is it that basically that like everyone benefits from that innovation um but if you're running a product that hundreds of millions of people use, you still need to run this somewhere.
And you still need to scale this service, like the scale that Google has and the scale that OpenAI has. I mean, it's no surprise that the DeepSeek website has been barely accessible for me, not because I'm in Italy, but like it was... Often down, the DeepSeek chatbot was not loading. I've seen people having all kinds of issues with the DeepSeek hosted API. Yeah.
If you really want to be the next model, you still need to match the arguably incredible performance of American models. Because they have a whole... Like, you know, set aside the fact that it's bad for the environment. But when you look at it as an object, it's undeniable that it's, you know, something that is scaling to how many hundreds of millions of people and how many...
billions of requests per day. From a purely web engineering perspective, it's remarkable, right? And so it'll be fascinating to see if the DeepSeek company can do that. I saw some reports saying that they are using these NVIDIA GPUs that are not the H100s that like... most companies are using are the H800s, which are smaller and not as powerful. I also saw some other reports saying that they're using some Huawei CPUs. It's a really fascinating infrastructure.
that they have put together um but here's the thing once again i think the the most important aspect to understand is that deep seek optimized something that already existed yeah I think, though, that the angle that OpenAI and Google will use to continue to justify their capital and operational expenses will be, well...
But if you want to push the bleeding edge of this stuff, like if you want to come up with the next new thing that then others will copy, we'll need the money and we'll need the horsepower. It'll be interesting to see how this shakes out. Like, is something like DeepSeek enough for most people? Or can open... I mean, OpenAI, you know, they literally are now... What's it called? Project Stargate in the US? Like that 500 billion...
Project, you know, backed by Microsoft, by Oracle, by the US government and others, like building a massive data center. Like that's a lot of money going into this. And that's why it is scary to have this little Chinese startup come up and say, hello, we have this and it's free and we spent less than $10 million on it. Goodbye. It's, you know.
It's interesting. It is. Yeah. And the timing really couldn't be worse for some of those people asking for billions of dollars. Like, I understand why they would be freaking out. Yeah. What is also fascinating here in this multiplayer AI onion that we just unpeeled is the apple angle. For a couple of reasons. Now, obviously, as we have established, Apple is at least a couple of years behind. Apple doesn't even have a large language model to begin with. And here we have companies
you know, doing reasoning models, O3, Gemini 2, deeps, like Apple doesn't even have a basic large language model in the first place. So setting that aside for a second, Apple also has a problem in China. because they want to roll out Apple intelligence in China, but they cannot because the Chinese government doesn't allow models like ChatGPT or Gemini. You know, obviously, you know...
This is the same as social media. You have to use specific services to be approved by the Chinese government. So there's a couple of interesting angles here. The first and more obvious one, I think, is that Dipsick could be the in that Apple was looking for. to be able to offer Apple intelligence in China with an extension. Basically the equivalent of the Chai GPT extension in the US and other markets where Apple intelligence is available.
DeepSeq could be the Chinese equivalent for it. A Chinese company with a large language model, high-performance large language model, could be the next extension for Apple intelligence to be approved in China. I could see that. It would also be fascinating to have another approach and to see Apple acquire DeepSeek. Now, Apple doesn't often make big acquisitions, but DeepSeek is a small company. The talent...
You know, DIPSEC is a hedge fund. They have a small team of engineers working on this. Again, I saw some reports saying it's less than 50 people. Who knows what is correct these days anymore. But it would be interesting for Apple to say, well, to kickstart our own large language model, maybe we could acquire this. And it would fit Apple's approach of like, it's high performance. It's more cost-effective. It's more energy. It consumes less energy than an equivalent to ChatGPT or Gemini.
it would be interesting to see Apple consider an acquisition for DeepSeek. Although I have also long thought that eventually Apple will acquire Mistral, which is the friend... the France-based AI company. But I do think that eventually Apple will need to acquire some kind of large language model. We'll see. I think more realistic, I could see a deep seek integration in China to get Apple intelligence out the door in that market. I think I pretty much covered it all.
Except, oh, there's one final thing that I think it'll also be something that we need to keep an eye on in the short term, which is the... Oh, there's actually a couple more things I want to say. The first one is...
I saw this article on the information about how a bunch of different businesses and what do they call it, Stephen? SaaS companies? Yeah, software as a service. Yeah, yeah. A bunch of these SaaS companies are like... jumping ship to the DeepSeek R1 API because it's much cheaper to run than OpenAI or Google or Anthropic.
Which, by the way, Anthropic, what's going on at Anthropic these days? Like, what are they doing? Like, I feel like we haven't seen news from Claude in how many months now? I don't even know. I think they're mostly trying not to get sued by content makers. No, that's... That's perplexity. That's perplexity. That's right. Yeah. To be fair, Anthropic is the only company that's actually thinking in a more serious way about safety. So that's commendable of them.
I do think, I saw some people say that, and this is where I disagree with the Apple community consensus. I saw people say, oh, actually, Apple being late to the game with larger language models. is a good thing because apple now can be an aggregator of ai and services and larger language models i think i cannot use the expression that i want to use on the show
But you can imagine what I want to use. I don't believe that. Because let me tell you, if Apple were not, if Apple hadn't been caught flat-footed in this space, If Apple could offer a large language model now, don't you think that they would? Yeah. And also charge for it with a more expensive and powerful version? You would think that if Apple could, if they were in a position to do this, they would say, nah, we're good as an aggregator. Let's, you know, that's our strength.
The app store is our strength. I mean, do you know Apple? Do you know modern Apple? Do you think Apple relishes the idea of being an app aggregator for real? And second... The whole idea of aggregation, if you put the value in Apple as being a distributor of apps rather than the provider of core services on your phone, You know what? Today, the more I use these AI tools, it's never been easier for me than it is today to switch from an iPhone to an Android device.
if I wanted to. And that's because all the work that I've done in ChatGPG and Cloud and Gemini, it's all based on a web service. It follows me around. And same with Dipsick.
I can use DeepSec on my phone. I can use DeepSec on Android if I wanted to. I can use DeepSec on Windows if I wanted to. So this whole idea of like, no, well, actually, it's not that Apple is behind. It's that Apple is an aggregator. It's that Apple is late to the game and this is good for... them because apple gets recognized as the app store i mean come on
That is like, you know, the classic tale of the fox and the grape. Like, no, absolutely not. If Apple could offer a larger language model today, they would. Now, all these AI tools... I think the greatest threat to Apple right now is not necessarily the fact that they are behind in large language models, because I could see Apple coming out late and offering something that is, you know...
powerful and much more integrated the greatest threat to apple right now is the web and the fact that all of this is web-based and web-based means inherently it's portable and you can take your data and you can take your identity, and you can take your projects, and it doesn't matter the phone that you're holding. It doesn't matter the computer that you're using.
The web is also the number one thing they should be worried about as a platform maker, right? I mean, just looking at the things on my dock, a whole lot of them. Just web apps and thin wrappers, right? Yep. There is a world that could be coming where being a vendor of an SDK doesn't matter as much as it does now. Yep.
I think this is something that I've been thinking about a lot lately. I think over the past five years, we were looking somewhere else in the Apple community, and we haven't kept an eye on what exactly... was going on in the web industry and in progressive web apps. And I think it's incredible what is possible today. And that's... You know, I think in hindsight, 20 years from now, we'll maybe look back at this and be like, that's a problem that Apple ignored. And it calcified into what...
became a big issue for them, which is developers and a whole new generation of young developers being fed up with the rules of the App Store and being like, you know what? I'm going to build whatever I want to build and I'm going to do it in Google Chrome. and in a web browser and on the web when there are no rules and the progress that has gone into the kinds of modern web apps that you can build today. I mean, look at Notion, right? Look at all these services that are now taken for granted.
in any workspace, and it's all web-based. And it's all web-first, actually. I saw yesterday a post by someone who was running DeepSeq in a web browser with JavaScript-based acceleration. I was like... yeah you know like incredible dude finn has it running uh on an iphone using core ml like yeah wild yeah so i think
The whole idea that I know actually this is great for Apple because it ties Apple to this identity of the App Store as an aggregator of the best stuff. Well, what about the Google Play Store then? like it's literally the same idea you go to the google play store like but uh but do you like i don't think of google as the google play store no that's yeah so uh we'll see what happens but i think this also also like yet another reason that's a bad take uh
What part of Apple's business has the most risk of being blown up by regulation around the world? It's the App Store. And Apple is clinging on to their model. We've talked about this a lot. They've clung on to that model at their own peril around the world. Like, what are we doing? Yeah. Yeah. So it'll be interesting to see what Apple does here, not just because of the threat of web-based products that...
It doesn't really matter which device you're using, and Apple is a hardware maker, the threat of open source. Like what happens? And this is... obviously extends to google extends to open ai extends to microsoft like what happens if if the best models
are not closed proprietary models, but they are open source models. Like what happens then? What happens to the economy? What happens to all of these investments? Is Silicon Valley going to continue to freak out? There, you know, anyone who's been...
following this space for the past few months will know that there's a curve of progress to the open source community and to the open source models that's only going up and up and up. And it's, you know... I can see why, you know, Sam Altman puts up a good face on Twitter, but I wouldn't, you know, did you see the report that, for example, at Meta, they have assembled multiple war rooms to figure out how exactly...
dipstick built the model and i can see that because it's it's um you know and meta is also doing open source obviously but it's llama yeah uh man it's it's it's been Even if DeepSeek, you know, the story ends up being that actually they had more GPUs than they said. Actually, they spent more money than the narrative initially said. Which is definitely possible.
Which is definitely possible. But regardless, what's in the white paper is in the white paper. And regardless, the approach and the engineering was clever. And so I think it'll... It'll have repercussions. It'll have consequences in, you know, it'll ripple out to the entire industry and it'll be fascinating to see. But it'll also be fascinating to see, since we are an Apple show, what Apple does here. And I am convinced that...
I think the most realistic option is that they will partner with DeepSeek in China. Yeah, we will see. I am sure someone at Apple is looking at it for sure. Oof, thank you for letting me do this. Yeah, no, thank you. When we talked the other day, you're like, I want to talk about this. I'm like, sweet, I'm not going to pay attention. Just like let you sink this information into my brain. So thank you for your...
your service. There's no denying we've had some rocky outros for the podcast recently. And our friend Rob has made a new tool. That lets you design your own outro for the podcast. It's called one, two, three outro maker. Incredible. Perfect name. Rob is the best. There's a link to that in the show notes. If you want to.
If you want to make your own outro at home, but for now, I'm going to give it a shot. You can leave feedback for the show at connectedfeedback.com. You can join at getconnectedpro.com and get longer ad-free versions of the show. each and every week this week we talked about task managers and i issued an apology because i messed something up on the show last week
You can find our work elsewhere. Federico is the editor-in-chief of MacStories.net, and he is Vatici across the Fracture social media landscape. Mike is not here, but we still love him. He's the host of many shows on Relay. You can check out his work at Cortex Brand. They have some new Sidekick products out this week. And you can find him, Mike Hurley, across social media.
You can find my writing at 512pixels.net. I co-host Mac Power Users here on Relay. It comes out each and every Sunday. This coming Sunday is our review of Apple Intelligence. We spent... a lot of time preparing for that episode. I think it's a really good one. So look for that on Sunday. And you can find me on social media as ISMH86. I'd like to thank OneBlocker for supporting this episode. And Federico, until next time, say goodbye. Arrivederci. Bye, y'all.