Also media, Hello and welcome to Better Offline.
I'm your host ed ZiT Tron. And of course you can go into the notes for the episode. You can take a look at links from a newsletter. You can go and buy some merch as always, but much more importantly, today I'm joined by the esteem Steve Burke of Gamers Nexus. Steve, thank you for joining.
Us, Thanks for having me.
So you did an incredible video on the black market smuggling of in videos AI GPUs into China, and then there has been a thing that's happened where the video has been pulled down due to a DMCA complaint from Bloomberg. What is happening?
Yeah, it's so the video, it's really cool video where we went all over China finding these GPUs. As part of that, we included a recap of all the news of export controls or the US controls GPUs that are above a certain performance threshold and the allowance to export those to China by American companies, and so that was the story we wanted to look into. Okay, but they're still getting to China and so how is that? And as part of that recap, go through all the news
we've got clips of you know, various politicians. There was a clip of Obama in there, and there was a clip of Trump in there, a couple of them talking about AI or GPU export control bands. Of course AI, I know ed is one of your favorite topics. Yes, yes, me as well. And yeah, so we received a copyright strike from small, small publication called the Bloomberg. Yeah. So that strike took the video offline and now we are
currently challenging it. And so the strike was for specifically, I believe it was minutes twenty two until minute twenty three point fifteen in the video.
Of Donald Trump speaking the President of America. Correct, Yes, that is ludicrous. And you went to New York specifically to have it out with them, did they Where do we stand as we speak, because this is going out in about a week from now. It's August twenty eighth, so who knows what twenty sixth even, I don't know what happened next.
Yeah, if it goes out in about a week, then that'll be pretty close to when we should have an update from YouTube.
Because the video is off right now.
Correct, the video's gone. And it's very interesting how copyright strikes work because if basically the way it works is a company files a strike. They decide if there's infringement of some kind, and they file it. YouTube then basically, especially for comedy, Bloomberg size will automatically approve it. It's then up to the creator to dispute that the videos offline. During this entire process, any money from the video is held at least in escrow, and so then you can
dispute it if you want. At that stage, YouTube then decides if it's going to accept your dispute. It accepted. Ours is written by a lawyer, so that makes sense, and then they step back and they're like, Okay, we're not involved anymore. It's between you guys. And so the party that filed the claimant, so Bloomberg here, has ten
business days to file proof of lawsuits. So they if they submit to YouTube proof that they are suing us, which thus far as we record this, they have not, then at that stage YouTube would keep the video offline and they kind of let the two parties resolve that amongst themselves. If they don't submit any proof of a lawsuit or they don't file a lawsuit and then show YouTube that they've done so, again, they haven't at this point. Then basically in ten business days the video goes back up.
We get fucked over for the ten days in between, and then the money, in theory should be released back to us, the ad revenue that was made from it, if it follows that course. But the biggest damage that's done is from the loss of momentum in those you know, total time period maybe thirteen days or so. But why would.
Bloomberg do this? Are they really that attached to videos of Donald Trump? Or is it something else?
I mean, I yeah, we talked about this in our video where we flew out to try and speak with them. I have a number of theories. We don't know their motive precisely, but you know, there's a few strange things going on where first of all, I've noticed that they have not claimed any or struck any that I'm aware of the re uploads of the video. So that's interesting because there's hundreds of them. It was just hours that
want top initially. Secondly, we have used that clip before and that hasn't been stricken and so specifically in this black market video. And yeah, the theories we came up with were there's some competing content from Bloomberg where they attempted to do what we did. It was really a failed attempt. They tried to find smuggled ai GPUs in China as far as they kind of presented it in data centers, they were unable to do so they couldn't get any access. They filmed sand in the desert. They
go home. There's also that's all they go. Yeah, they drive around at a desert, they point the camera at some buildings that are being constructed. They say, you know, there's ai GPUs in them their hills, and then that's about it.
It's kind of funny that you you appear to on your own and with your with your team obviously able to get this access to these large legacy media people don't, which it's meant to be the other way around based on what everyone says.
So what is it the makes your approach.
Different to bloombugs, Like why did you get stuff that they didn't?
I think rather than just the desert, right, Yeah, Well, to be fair, we did not film the desert, so they did have an exclusive there. But I think it comes down to the approach where we looked at it like this is an interesting concept, this idea of black market GPUs, But from my point of view, we approached it as well, let's just see what are the people involved in this chain think and do they think it's a black market or do they just think it's a market.
And a lot of the people we worked with were just, I think, amused at the concept of being involved in the media process and wanted to see how it works. And so I honestly, I think a lot of that access, especially to the people who are just sort of like normal people. They're not these big, you know, executives at some huge corporation. They're they're middleman and they're people transacting
GPUs on the ground. I think for them, if you go into it without this free supposition of some kind of particular i don't know, leaning or ethics of whether what they're doing is or isn't okay or whatever, or.
Even expecting what you'll find exactly.
Yeah, I think if you go in with an open mind of like we're just gonna hang out with this guy for a day and just observe his job, people are you know, they're pretty happy to share that.
So, yeah, how did you actually start those? So you you came up with the idea that you wanted to see how elicit GPUs making it into China. Did you have contacts in China or Hong Kong and so on that you talk to in advance and they kind of say, oh, if you come here, we can show you this. What was the process?
Yeah, the process was really cool. We're still kind of learning how to iterate on this, but each time we're The big thing we learned this time for this investigation was really just trying to that first contact matters a lot, and being able to work with that first person we find to then find their contacts and kind of follow the chain down the line. And so the first guy we found is a professor at the Chinese University of Hong Kong. He buys and uses these high end GPUs
in servers for educational purposes and research purposes. And as soon as we made contact with him, we were able to work with someone who first of all speaks you know, native level excellent English, so that certainly helped. And then secondly who has a lot of local knowledge and contacts. And I think a lot of it is just kind of knowing which, in our case, which link in the chain you need next to make the story makes sense. So in his case, he's a user of these GPUs.
So next we need is some kind of supplier, you know, and then we need their supplier.
Yeah, so what it's all quite complex. I did watch the video and it was it was what three three hours or so or more even really wonderful. And so what exactly are the GPUs that are restricted? Was it just aigpus? Was it like consumer level ones as well? What is exactly being handed around?
It's both. So on the high end side, there's some smuggling and presence of these data center class GPUs, so that'd be things like EH one hundreds, Hopper one hundreds, A one hundreds. We actually physically saw several of those and things like that. On the consumer side, there aren't as many restricted consumer GPUs, but there are a few. So the RTX fifty ninety is one of them that has thirty two gigabytes of video memories. That's very valuable
for these use cases. The ARTAX forty ninety actually is another one, and that one's kind of amusing because it's so GPU right, yes, consumer GPU, yeah, and last gen consumer GPU at that, but it's easily modified to carry double the video memory as the actual official end video spec and the skew so they can increase it from twenty four to forty eight gigabytes of memory, which is extremely valuable for these training and LLM type tasks where it may be a last gen So it's technically a
little bit slower, but if you can fit it in memory, you can still run it, versus if you can't fit it in memory of the model you made, not be able to run it at all. And so those are the consumer cards the forty ninety. That was interesting because one of the guys we spoke to who is effectively a buyer and seller of these band GPUs, he was asking me on camera. He was seeking confirmation of well, but are you sure the fifty nineties banned? Yeah? And I said, yeah, I'm sure. Here it is on the list.
And he was so confused because he said, but they're just all over the market next door. We walk over there and there's dozens of them.
So it's so strange as well, because it doesn't seem like you basically walked off the plane and had a price sheet from just a guy, Like it wasn't clear how you got the price sheet exactly. Perhaps you can't say, but it was just Yeah, these things a band other than the fact we have just like guys who sell them literally everywhere. Yeah, getting in.
So there's a couple of ways. The one of the ones that we showed in the video is really interesting. To the very end of the process. We actually this is one of the reasons we delayed initial publication. We were able to, with help of a viewer, locate someone you could actually classify as a smuggler in the US. So someone from China in the US.
And.
He buys GPUs from people in the US. He drives around all over and buys most through forty nineties, which are in a high demand in China, and then he'll strip those downs that could be removing the cooler. I don't think this particular guy disolders them. I've heard stories of some people desoldering GPUs and mailing them that would
reduce your shipping costs a lot, increase your margin. But he buys the forty nineties ships the boards back his profits about three hundred dollars US per board before taxes. It wasn't a one hundred percent clear if he does or doesn't pay taxes. And then at that point, once they are either shipped into Hong Kong or Macau directly if they feel gutsy enough, or indirectly through a third country, a middle country where there is no export control, you know,
then they arrive and they get redistributed from there. There's sometimes hand carried as well, like by students, international students who just fly back with a fifty ninety. They bought it best Buy or something, and they can double their money roughly if they're lucky.
That's so, it's wacky that there is such a big black market, but I guess the export bends just created it, as did this whole AI, this crazed AI moment. Did in fact, while you were over over in Hong Kong, did you actually go to China or can't remember? It was just in home? So did you? This is an anecdotal thing. How was the pressure of AI there? How big was the conversation the advertisement? Did you see it on the same level? It's fine if you didn't, I'm just curious.
I saw it more in Taiwan, which we went to as well. So yeah, after the China trip, we went over to Taiwan, and I don't think it made it into the cut, but we shot a short clip of just a bus, like a public bus with a gigantic
AI the laptop advertisement on it. So I would say it's kind of at least there, it's similar to what you see here where it's you know, if you're anywhere in the vicinity of a best Buy or a city that has technology dealers, then you're gonna see AI ads In China, Yeah, I saw it in the tech markets for sure, some banners, but it just it. I don't know. That's I only understand Chinese if I'm actively trying to
understand it, you know. So it's I think a lot like it's filtered out for me, right, Yeah, I was good.
Yeah, I assume you speak some You seem to speak some Chinese. I can't speak anything but English, so I could not correct your Chinese, doll.
You do speak more proper English than I do, though, so I'll.
Get Yeah, yeah, I speak all kinds of it. These days. It's not going well. People criticize my accent all the time. So as far as how the government sets these limits, one thing he did really eloquently towards the beginning of the video as well, was he kind of went through how arbitrary the limits seemed to be and how in Video seems very capable of adapting around them to the point they almost doesn't make sense.
Yeah, yeah, they they've modified the regulations a few times. And it's it's so this particular part I don't really blame and video for for this, which is just trying to you know, the government sets some regulations and a company says, okay, we'll comply with these, and then they comply with them, and then the government kind of says no weight not like that.
Yeah, that's very standard for pretty much or regulation. It maybe it's good or bad what have you, but it's not. It happens all the time with companies.
Yes, And in this case, one of the things that as the I guess AI world developed and the applications, one of the things that changed was this understanding that memory capacity is basically the most important aspect of it. And the initial versions of the formula didn't consider memory in any scenario. What I remember, it was based on the marketed flops of performance on the spec sheet multiplied against the bit length of the operation or something like that,
and they later added memory bandwidth. Memory capacity though is still as far as I'm aware right now, is still not considered in the formula.
Yeah, it just it feels just arbitrary. But I imagine also it's quite difficult to find a way to do this, especially when the people setting the regulation might not have the best handle on computers.
Yeah, I think I think the best way to do this would probably be if they wanted to and I'm not I don't have a particular one way or the other, but if they wanted to control it, I think the best way is probably some kind of benchmark or set of benchmarks, just like you would do for a review, where you know, you say, okay, if it exceeds some performance threshold in this real world use case or multiple of them in aggregate, then it's control, it's export controlled.
But instead they just kind of calculate based on the spec sheet and the companies that make these things. Obviously, you know, full appropriate credit to Nvidia. The company is extremely confident with making GPUs. They know what they're doing, and they're going to know a lot more about how to tweak those dials to comply then a government agency will so.
So on the subject of Nvidia knowing things, do you think that do they seem aware of any of this, like any of any of this smuggling happening. Do they seem to Did you get any comment out of them?
We asked a lot of the people in the chain what their perception was of Nvidia's awareness, and so the professor at the Chinese University of Hong Kong, he had a great answer when I asked him, do you think Nvidia knows? And he kind of he paused for a second, and then he said, how could they not? You know, like these things are very expensive. You've got thirty plus thousand dollars GPUs and it just seems like they would all be tracked. So what was the term?
There was a specific terms like kind of like basically equivalent of ignoring him.
Yeah, so the Chinese has done eg and b egn, which means open one eye, close one eye, or turn a blind eye? Is the idiot?
Yeah, And that's awesome.
It's just very cool. Yeah, it's super I mean the coolest part about that was multiple people said that exact idiom and it's it's just it's such a specific choice of words. It'd be the same as if you asked like three people a question, you know, they all said, oh, they turn a blind eye. At some point you're like, Okay, this seems to be a really consistent belief here, But yeah, I think from a video standpoint, it's whether or not they want to actually control what gets in. It's hard
to say. Certainly, from a pure financial standpoint. The more gps they sell, the more money they make that they probably don't really care who they sell them to as long as they sell them to someone and then where they end up after that. I mean, maybe to the extent that video doesn't want to piss off the United States government. They you know, they try to comply as much as they reasonably can. But how much do they actually try to control ingress into a country that's export controlled.
It's hard to say, But certainly I think they are aware of this happening, and I just I have a really hard time believing that they are unaware their gps are making it into China when they shouldn't be.
And I imagine it must be quite hard to stop. Like I hate to have any sympathy for in video, but it does. It does feel like something that would be kind of a moving target. Because one thing that was consistent across the whole video was the ingenuity of
the people involved. It does make you see like American entrepreneurs sometimes and they're like, oh, yeah, it's tough to you know that the hours are very tough, and you know, I was covering all not and all of these guys were just like, yeah, I've got a thing in the back of my car full of wires. Yeah I've got I have a whole rig.
Yeah it's crazy, and that is I mean again, you know, from that particular side of things, I don't suspect. I mean, look, it's a government regulation, and if the government wants to regulate it, it's kind of up to them. Obviously the companies need to comply with it, but I wouldn't expect a company, even Nvidia, to send in any kind of enforcement.
Maybe what they would do is if they wanted to enforce it, you know, they might identify, Okay, this purchaser, there's evidence of this purchaser exporting to China anyway, and so we're just gonna stop selling to them or something. But that's kind of I think the most you might expect of them. But yeah, as far as the ingenuity, it was pretty crazy. Like the guy in the US who buys the cards, he he just got a Prius with a massive like battery bank in the back, like
a ups with a lithium ion battery in it. He's got a test bench hooked up to it, an ATX test bench. Pops the trunk open. He's got a spare license plate in the trunk for some reason. I don't know why. Uh, that's not to ask. Yeah, I don't need to know. And uh yeah. And if you want to sell him your card, you're just some American new found him on Facebook Marketplace. He at the time we were talking with him, he was paying two thousand dollars flat.
He makes three hundred when he sells it, and you bring them the card, meet him wherever he tests it, and in the back of the you know, in the trunk of the car, he just runs a benchmark, make sure it's good, and then buys it and ships it back or carries it back, depending on how secure he feels.
And then also to your point of ingenuity, the shops that are capable of modifying cards, where they're just they're repair shops, and they just happen to be really good at board level BGA or ballgrid array device repair or
or swapping, so they can pull memory modules. They can add memory modules, swap the GPU between PCBs, and they're able to keep Silicon in service where you know, they're just repair shops, so it's kind of like from their perspective, look, they're not doing anything illegal in China, they're just keeping
boards in service. But for people who need these high capacity GPUs for AI tasks and they having to be export controlled, a shop like this would be capable of taking a card that has died, you know, through through use or just needs to be upfit to be better, and they can do that for pretty cheap and basically in a couple hours.
And it's not illegal to sell them in China once they are there, right, They've like that is not a like China is not prosecuting people for any of this.
Correct that it is not illegal to own or sell these GPUs or buy them in China. It's legal if you are an American company, if you're Nvidia or your best buyer or a partner or whatever. It would be illegal to sell the export controlled cards to a company or an entity or person in China. But once it's there, yeah, there's no real control over it. The only the only place that we found that China steps in is if
there's some kind of undeclared ingress through port. So like, for example, there were CPUs and GPUs at various points in the last several years that were smuggled in with creates of live lobsters or prosthetic baby bombs.
But was that real?
Was the lobes the thing real?
The lobster things real? Despite what Nvidia said.
Hell yeah, I love the So what actually describe the full.
Loves the situation? I think everyone wants will want to hear it over lobs.
Yes, So the loster situation as it were is basically there was a Hong Kong uh port, so customs intercepted a white van I think it was. There's like a customs post they post photos of the things that they are able to snipe being brought in improperly. So Hong Kong's a free trade port, but there's still certain declarations you have to make and sometimes people won't make those. And so there was a van bringing in crates of undeclared live lobsters driving on I think it was the
Macau or the jew Hai Bridge or something. But anyway, they got intercepted and alongside the live lobsters were GPUs and they were older, so I think they were quadros, but they're brought it all being brought in improperly. And yeah, so customs post the thing. They're like, we found this, you know, and I think that's just kind of like the here's us doing our job and maybe scared people
off or whatever. And then later AI startup Anthropic released a statement talking about how there needed to be more strictly enforced export control in the US, and they mentioned smuggling of GPUs with lobsters, and Vidia responded and I don't have their statement in front of me right now, but it was something along the lines of it would be better if American you know, AI companies would focus on innovation rather than the part I do remember was
tell tall tales of prosthetic baby bump smuggling or GPU's sensitive electronics being smuggled with live lobsters. Turns out that is not a tall tale. That is a thing that happened.
So, yes, this is the thing people say. Jensen Wong is cool. If you're cool, you say, yeah, well, our chips like, we don't like this happening. This is bad, but doing everything we can't stop it. But yeah, they're doing James bondshit to get our GPUs, Like, come on, you could just say that, like they do seem a little sensitive, and we're we're recording this the day before their earnings, and I got no idea what's going to happen, No one does. Everyone's a bit everyone's a bit worried
about it. So it's I think it. The China market is going to get extremely interesting because they've got the H twenty band. But now the H twenty band has been lifted, and now we're in this weird spot where I don't know if you've heard anything about this yourself, where there are suggestions that the Chinese government is actively saying not to buy in video GPUs. If you heard anything about that.
Yes, So the Chinese government, I believe it's called the Cyberspace be a security agency or something like that, but one of their government agencies put out a statement following separate statement from a state owned or at least state you know, controlled in some capacity media report where the statement said or at least floated the possibility that Nvidia's H twenties may have some form of spyware in them.
And so the suggestions we're tracking or government back doors, and so that was kind of the accusation, and Vidia of course posted a statement saying absolutely not, this is not a thing, and you know, just to be fair here, there's not currently, as far as we're aware of, any firm evidence of this. But I'm also sure that governments know things about you know, about products from other nations that we would never know anyway, So who knows. But
that's where it stood. Nvidia said, we don't have these. China's agency said to companies in China that they should be wary of using H twenties and advised against purchasing them for fear of some kind of backdoor or spyware.
So how realistic would it be for Actually, did you hear any scuttle about around this of switching to Huawei chips or something?
We did ask? Yeah, I think we might have only kept one of those questions in the actual video, but I did ask a lot of people Huawei from the users' perspectives, the people we spoke to, they none of the people we spoke to are using Huawei components right now, so they're getting more powerful from what I understand, I think there's a few hurdles. One of them is. It's kind of the same reason where the companies aren't even really using AMD and they've been doing this for a while,
whereas Huawei is pretty new. And a lot of that comes down to Kuda, where Nvidia is the proprietor of Kuda, which is a library they can use to accelerate tasks in which a lot of software is written to use, I mean even just rendering a video.
It's the coding language to get accelerate computing, right, correct, Yeah, specifically on Nvidia GPUs.
Though, yes, it specifically works on Nvidia and nothing else, and so AMD would have to use i don't know, open cl or something else, depending on like for video encoding, you maybe use open CL. But that's the biggest limitation where a lot of the sort of so called AI tasks are reliant on Kuda to operate at any reasonable speed. And so until there's a big push to move away from that, it's Nvidia GPUs all the way down, you know. So,
and maybe the push is export control. Maybe the groups in China that need a higher volume of GPUs than they're able to easily get of actually say okay, let's just let's put all of our resources on trying to use open Seattle or use something out of or PyTorch or whatever, so that may be the direction that goes.
Yeah, it just feel it does actually feel like Couder is the thing that needs to be broken here for better or for worse. I'm just saying that there's a lots of talk about oh, making more powerful chips here and there with AMD or Huawei, but it feels like couder really is the thing that they actually need to change.
And I guess the probably explaining to the markets what Couder is is difficult enough, but it feels that that if they could move past Kuda, that would be the actual thing that we're doing.
Yeah, it's kind of like the X eighty six situation where getting off of X eighty six.
Just for the listeners, what do you what do you mean by that?
Yeah, So your average computer is very likely an X eighty six computer, which is a specific micro architecture that has to do with how software runs and interacts with the hardware and the other so around the system. And so it's kind of like a prerequisite for you might have an application that requires Windows to be compatible, and
this is another one of those types of foundational prerequisites. Okay, X eighty six as opposed to for example, ARM and ARM laptops made kind of a splash a while ago because they were trying to basically assert that, hey, look we can run stuff that typically would only run on an old X eighty six micro architecture and now we
can run it on ARM. And so it's that is something where Intel an AMD, although mostly Intel have a vested interest in keeping everything on x eighty six, and likewise on Vidia has an interest in keeping things on Cuda.
From an end user standpoint, there are benefits to this where Kuda, for me, for rendering a video, it just works better than something else, right, right, And so it's kind of it is a chain or the egg problem where it's it's kind of like, well, okay, you know, on one hand, you don't want a company to develop the monopoly only because there's some kind of anti competitive practice that's keeping them in place. On the other hand, you don't want to punish a company either for inventing
something that's just better. And that The problem we run into with software is the developers need to choose what to support, and a lot of times they choose n video and maybe not something else, because if you have limited resources, you're going to go with what ninety percent of the market has. It's kind of a self fulfilling, you know, it to feedback loop basically. So to expand it to something like Huawei, they would have to break
that kuda. I don't know if you want to call it a moat or something like that, but or a walled garden I guess either one of those or a walled garden with a moat around it. But they would have to break that down to really become more viable and increase the performance.
So before we go, did you catch any black Well out there or any signs that black Well would even make it out there?
Yeah, So in addition to the fifteen nineties which are Blackwell, as far as like the high end server stuff, we did work with. So there's two companies we worked with here. One of them we physically visited in Taiwan, and there's
another one we were talking to in Stapore. And what we learned is that there is some intermediary basically passed throughs that can happen through for example, a Taiwanese testing agency where a Chinese company might ask them to purchase Blackwell servers or high end Hopper servers and bring them into Taiwan under their company performed testing or pre testing on them, assembly, whatever, packaging, whether it's a facade or not, basically the stated purposes, make sure it all works, and
then once it does, forward that shipment to their customer, the Taiwanese company's customer in China, and so that would allow them to potentially get into China. And so in that situation, yeah, we were showing a room filled with for example GB or Grace Blackwell servers that were being tested and prepped.
Yeah, well I'm gonna I think we should wrap it there, Steve. So, the status of the video right now as we record this is it's offline. But by the time this goes live, you know, in I think September the third, it will go up. Hopefully it's back on. Where can people support.
You gamers and access just the YouTube channel is the best place to go. And yeah, hopefully it's back up within It should be within a couple of days of you posting this, unless Bloomberg submits a files a lawsuit, in which case they'll hear about you. Everyone will hear about it.
Oh and no hear about it from me as well. This is utter bullshit what Bloomberg is doing. It's ridiculous. But you have been listening to Better Offline. I'm at Zetron Steve. Thank you so much for joining us, and yeah we can do something in a couple months as well, but always push to having you ma'am.
Thank you right, thanks so much.
Thank you for listening to Better Offline.
The editor and composer of the Better Offline theme song is Matasowski. You can check out more of his music and audio projects at Mattasowski dot com, m A T T O S O W s ki dot com. You can email me at easy at Better offline dot com or visit Better Offline dot com to find more podcast links and of course my newsletter. I also really recommend you go to chat dot Where's Youreed dot at to visit the discord, and go to our slash.
Better Offline to check out our reddit. Thank you so much for listening.
Better Offline is a production of cool Zone Media. For more from cool Zone Media, visit our website cool Zonemedia dot com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts,