¶ Intro / Opening
Bloomberg Audio Studios, Podcasts, Radio News.
¶ The Vision for GPU Futures Market
Hello and welcome to another episode of the Out Thoughts podcast. I'm Tracy Alloway.
And I'm Jill.
Wasn't thal so Joe. We're still continuing our series. Recorded from the live show in New York. We had a bunch of great conversations. A couple of them were building off of discussions that we had had previously, and one of those discussions was in Chicago at another live show about six or seven months ago. Back in October, we spoke with Don Wilson of DRW about the trading environment, but also about his new venture.
Right and so his new venture is one that actually there's quite a bit of competition in and quite of excitement in, and it's essentially like okay, GPUs. We know they're very important for the AI boom, et cetera. The question is can GPU capacity, which is scarce, can it become a tradable commodity such that I can buy futures to lock in my price of access to compute power.
Could I resell those futures? Will there be speculators speculating on the upper down price of like an H one hundred running an H one hundred in video chip for an hour. This is a big question. We know there's a lot of interest in the actual compute, but whether there's interested in compute futures. It's tradable instruments is very TVD.
Yeah, And the analogy that everyone always uses is compute is the new oil, right, so why can't it have you know, a market structure that looks somewhat like the oil market. And there are challenges. Fungibility is a big one, like one chip might not necessarily be equal to another chip or one ship.
The same chip at one data center might equal to the same chip at a different data center exactly.
And so even if you're not interested in AI, what I say here is like the market structure questions and the idea of building an entirely new market is really fascinating to me, and I think others will find it interesting too. And we really do have the perfect guest. We're speaking with Carmen Lee. She is the CEO of Compute Exchange and Silicon Data. These are the two companies that Wilson is invested in, and they've already announced that they're doing futures with the CME. So really the perfect
person to speak to So take a listen. Last October we spoke with Don Wilson of DRW fame and he was talking to us about his new project, which was basically building out this compute exchange. Now we're here with you six months later. You're actually the one leading it. How far are you in this endeavor? And remind us what exactly are you trying to do here?
Yeah, so thank you for the the quizing one great audience. Before I do that, I actually going to call back to six months ago in the DOM podcast you did. You asked them question, what if compute prices keep going to go up? At the time September October, compute prices were going down across our chips. Now see what happened. I think you called it. I think you called the market called it. So I'm the fund of CEO for Cilicond Data, so that's the index provider for GPU indcees.
We recently announced partnership with CME, so we all been launching gp futurning options ME in a couple of months, pending the FDIC approval. Obviously that's quite exciting. We've been working on GPU indices for past two and a half years, starting twenty twenty four April, so it's been a while, and we launched for world's first GPU inducees at Bloomberg Terminal in twenty twenty five. A year later we launched
the partnership with CMME, so it's quite exciting. Separately, I heard you mentioned Compute Change before, so thank you for doing AGIL. I'm the CEO for compute Change, which is sport marketplace for GPO procurement. So we do reserve contracts for contracts as well as ree for pitch contracts.
¶ Market Participants and Hedging Needs
Let's talk about the variety of options that we have to financialize compute and so forth. So this, I mean, this came up in our conversation. The first conversation we had with we had with Ian Dunning. Who is the type of buyer who would want to buy compute on a spot market because right you talk about typically we think it's like these multi year contracts where some entity enters into a contract with a data center or a new cloud whatever, and this they have this for a while.
So who is the market, who is the buyer or the user of these instruments that might want to buy spot compute or very short term short dated compute futures.
It's a great question. So the compute market right now, for comput change, we have all our provider mostly are neo clouds around the world. It's one side. Another site is a big variety AI start up. So even though they are start up, millions of dollars on GPUs already. There are enterprises who are traditional businesses, but they are needing a note to notes, a few service here and there for their inferencing or I don't know, other deployment needs.
They are providers. They are influencing providers right they don't own GPUs, but they provide open source, open weights model support for other use cases. So what see big variety. Most North American firms they do a variety of combination contracts. Obviously on demand give you the most flexibility. You don't pay and you don't use it. However, you're also at
a mercy of defense. Apply curve at a given time, so translate to your price can go from three dollars to six to nine depends on demand supply curve shifting, so that doesn't help when you can have a predictable margin. And also in terms of scarce, you're not guaranteed to
feel GPU resources for next hour or next month. So you see a lot of people shifting from on demand to reserve even four contracts right, so full contracts you back the lock in a deliverables for next whatever month, right, starting in September maybe or starting no member of forbare weapons right. So this all comes because of market condition. So computing cover that the physical GPO procurement also token,
so we love to talk about token as well. On flip side, who's going to use the future options can be a similar set of people.
Right.
You look at oil market, which we all love double TM brand. Right, the people use double TM brand. A lot of them are naturally long oil. So the shells ran the producers. They need to hatch your revenue volatility by shorting futures or port options if you're naturally short oil americandline right, they want to control their cost volatility. They want to obviously use future options as well.
Simple to the.
Compute your new cloud or you're at anyone have the servers. Ideally you want to have predictable revenue streams.
So the neo cloud would be the shell on this example.
Exactly, you have GPS, right, all the banks were GPUs on your balance sheet right your long GPUs, then naturally you want to make sure revenue right is stable to certain degree, and then you want to use future to do so. If you are naturally short GPU, which is everybody in this room, amas you tell me you have GPUs right, then you depends how much you use. If you want to control your cost volatility, you want to use future to hatch as well.
¶ Ensuring GPU Quality and Performance
Just on the compute exchange side of things, if someone is buying like off the spot market, how do you guarantee I'm not sure quality is the right word for this, but how do you guarantee they're getting what they expect?
This is a great question. So I canna flip to slide if you don't mind. So I usually don't let use slides, but this time because you mentioned really good questions. So we actually call it GPU lottery. So we published a paper earlier this year at GPGPO conference with Jefferson Lab on GPO performances. Well actually, so well we can have your quit link, you know, to the audience later on. We actually this is a one hundred by the way, I know we didn't put tang on the day. Is
a one hundred forty gigabytes memory bandwidth. We prove there's thirty eight percent performance variants for the same trip, and we decompose it into the chip self, into provider and inter provider. And there's many reasons for that, right and to your point, you can you don't know until you get your GPS. We have a PLAT for GPU. CARFAX for GPU depends how to look at it, so we in compute change. You actually verify the GPU before delivered
to you. So basically you are you are. You CANFQ for say, hey, I one A two hundred BI two hundred notes. Obviously it will give you specs back and the commercial back same time, independently verify the performances on flops, memory bandwidth totals another information as ACE and other things. And as a user you can decide is price your most important criteria? Maybe it is, or maybe you're willing to pay a premium for dual location or the performances
that you care more about on latency. Right, we believe gave people the option and transparency it is the most important thing.
¶ Developing the GPU Price Index
Let's stick with the oil analogy for a second. You know, there's a few benchmarks that we all know about. There's brand. There's WTI, there's others, but those are the two that we talk about. If we transpose this to chips for a second, okay, we say you have an H one hundred index. We did an episode of the podcast last week, I think with the CEO of Servius, which is another
amazing company. Yep, yeah, but there are different Another type of chip for inference is your assumption that these indices are going to be close enough to the cost such that if you're okay, I'm running inference maybe on some service or TPUs or training whatever, some of these others, that an H one hundred index will be good enough as a hedging instrument.
This is the whole goal for me sitting here. Actually, right, there's a meaningful every financial products in the functional reason for commodity it is for hadging. Right, This speculation is great, but really for people to hatch their relativity, to do risk allocation, to do risk transfer, and then asset capitala location. If we can't do what you said, then we fail at our job. Right, So that's why we when all the way back the way we cut we develop our
index model is not a simple math. It's not Hey, you have two h one hundred to simple average, right, because then then you compare Apple to oranges, the two h one hundred can have different CPU different RAM defend this differential location depend memory, bandwidth. You cannot do simple math. What we do is we usually collect six months of historical training data from over one hundred data sources, and
we see which factor to have the price differentiation. So every day over one hundred and fifty thousand traded prices in just our platform, and we normalize the traded prices based on the different characteristics of the model itself and normalized to a base case. And then we do the math of settlement price calculation. Right, So then this price will be highly correlated, ideally as much as it can
to the price you pay at a new cloud for example. However, it won't be the same just like basis trading, right, like every other commodity is a basis risk. We're helping client colpeling the basis risk. So you know, hey, you us east, you may be BIPs higher or two than there's expectation a manageable correlation understanding of the indicies.
You mentioned volatility just then, I mean the reason people need to hedg just because of volatility. Are you seeing enough of that in GPU prices that like, this model makes sense because if it's just a steady line up or steady line down, like it's gonna be a kind of boring market.
So it's interesting. So last year, when GP prices are going down, the big conversation is why do you need indicies for something price will always go down? And this year is why do you want to inducease when price always go up? Literally is all the questions.
I get it.
It's pretty fascinating. So when we will look at volatility, we look at daily bomb volatility movement, not the price up and down. Right, the daily volatility for eight one hundred h one hundred is around twenty to thirty. So it's a very healthy commodity volatility range. So I don't manage volatility. It just happened to be the volatility that can change. It's all because we normalize it. If you look at each individual chip configuration at differential location, the
voltity are different. There's some chips with eight percent volatility, some chips with over one hundred. Because normalization of indices, you actually get very healthy twenty to thirty daily bomb.
¶ Price Data Collection and Market Liquidity
I'm always fascinated by like, you know, we look at the Bloomberg terminal, for example, and there's a price on the screen and it's just there, and we started taking for granted that like it had to come from somewhere, and maybe some commodities have like a you know, there's an existing exchange and a public price, and then there's also a lot of commodities just bilateral traits. What is the actual process by which you collect the most recent data?
So if you say, okay, an hour of h one hundred usage costs x right, whatever it is right now, how did you assemble that number? How did you gather that information from, say the inference providers?
So it is a very can be lengthy, depends on what data sources, the nature of GPU spawn markets conpictures just one of them, and then many of my in neoclaushyperscular marketplaces all have very different contras, size, durations, backs, and their way to manage their data right, So it's a lot of licensing, conversation, negotiation well and also context love myself, I don't know, I was used for boomboard data, so I was in data basiness for period of time.
So everything is pretty intuitive to me. It's very important to get a variety of data sources, especially for computing.
Like do you call them up? Like so it's like, okay, the price is different on the sum them up.
Well, you first of conversations say hey, I love what you do. You're bring your cloud? Can I license your data? And usually your feedback is what is in for me?
Right?
And I will tell our commercials And then your concern could be, hey, you know, if I give you all my data, I give my way all my secrets and I will go through traditional licensing agreement. What can I disclose? What I want from you? What I do not want from you? What's the pipeline look like? Are you right? You use a street market job? You are running my API to yours? Are you running into mind? It's a
lot of conversations. It's actually pretty standard conversation. And right now with eight million pricing points globally around two hundred data sources, it's pretty much bau A lot. People will say, hey, always bring up you can I have your data? It's always my ending.
You know, we were talking about GPU indices and you're not the only one doing GPU price indices for.
Sure, not anymore.
Yeah, not anymore. But when you look at some of the other ones, like sometimes they show different numbers or even different longer term trends, what accounts for the discrepancy there? What are you doing differently or what are they doing differently? I guess.
So I can't come on other people's mythology because I actually don't know different raw data. Different mythology will eventually draft different prices. So the way I would look at this is, you know, it's always smart for anyone to look at multiple data sources and then figure out what is the actual decision you have to make, which datasps do you trust? The market always volte whilst us things start trading. The market will always gravitates what things actually
help them hatch? Right, if you easily manupulatable, if you are not data source people acting actions, do you hatch? There was the point a sideh one speculation, right, So you know I'd love to say, I mean, I also strongly believe what the best but again I will let the market decide, which will happen very soon.
So of course, like yes, there's the economic rationale for the existence of a hedging instrument, and we can understand that someone who is an entity that from time that needs compute their short implicitly short GPUs, they want to hedge, et cetera. But the liquid markets also really do need
speculators and they need people betting on price. What are you seeing right now in terms of traders or institutions, et cetera, who economically can take both sides of the trade and how active is this getting where it's just a compute trading desk that is separate from their economic needs.
The conversation has been going on for a very long time with various banks, various multi participants, speculators. They are very excited. So some banks obviously have those both sides of the trade right, so they can cross off some positions internally. That's great. Always some they have to use leverage external products. So that's where we come in. The way I encourage them to do is I selfishly, I want them to start trading desk and compute. The more
people trade, the better for me, right selfishly. The same time, it is important for people understand GPU trading. It's not like you can just move someone from oil electricity with no background context jobing to GPU compute futures. There's a lot of context where number one GPU it is not homogenous product number two. You have to understand the use cases of eight one hundred h one hundred right now, they are not that correlated. Is that right? Maybe that's
not right. I don't know. There are use cases which they're pretty separated, but maybe their use cases they can be transferred and also their software layer to this. Right, so right now you can art give sooner use cases some large amount of models cannot be deployed and the legacy chips, but doesn't mean six months later you cannot do so. As the software layer compression model compression gets better,
optimization gets better, things can change. So really understand not just the hardware configuration, this local supply demand curve for the service self also software layer that's kind of critical, right, that's really changed the supply demand curve and all the way to the user behavior. So it's all it's going to take some times as we have engaged with a lot of participants make sure they have the right set up.
¶ Futures Settlement and Trading Access
I have what is possibly a dumb question, but the compute futures, how are those actually settled? Because I have like images in my mind of taking physical delivery of like maybe one of those big.
Okay, that'll be fun. So for the CME, futures will be financially settled just like the traditional oil settlement price. That priceis goes four contracts. Well obviously we do four right now, I can fut change, but we always open to do you know, physically deliver futures, especially given we do have silica mark which is GPO benchmarking. So imagine the future. You can do, Hey, I want to twenty grade A B two hundred, this configuration, this shape of servers in US East and then at the end we'll
get that. Well, usually API costs so you don't get physical and it's not as coo as physically give you a way way for but you get APIA costs.
One can dream.
How do you literally trade it is in like let's say there's probably some very bright people in the room now with an institution. When it's all listed in everything, does it need to go through like a future's broker? Is it like a could it be like a prediction market if you just go to a website? Like what is the actual How does someone actually get in this setting? Aside whether they're sophisticated enough of whether they know what they're doing a lot of people trade to have no
idea what they're doing. Yeah, setting all this side. Yes, you know, only trade what you know. But like, what is it through a prime broker? Like how will people actually be able to participate.
In this market?
The beauty of CME is you can do the same thing you're doing now treating semi products. Okay, the same process and processing margin. That's why you get great margin optimization. Right, everything is BAU it's no different. We don't have anything right now.
So any commodities broker that someone has, they will be able to on that platform. They will have access to these.
Instruments exactly right. Yeah, we make it easy for people.
Would you be upset if a prediction market set up a GPU price contract of some sort with that into your business?
Not at all. So we actually work with poly market. Last year someone actually listed my product at polymarket with my consent. It's pretty it's always start like that. And then someone told me that, and we try to Polymarket and say, hey, do you want to do something you know more real? So we did. Fabruary settled and April settled a few contracts on polymarket. I just with test the water right obviously we're exclusively with CME right now. But yeah, so I think obviously you have to do
it right, licensing, nonminal, pillared, it all the right things. Yeah, you know, I don't. Mark can do whatever they want, and then people will choose the best product for them to use.
¶ GPU Market Trends and Refurbishment
Sitting aside the financial instruments for the moment, would people think about AI and they think about the use of GPUs, they mostly still probably in their mind think of like open AI and Thropic and Google basically, and that's kind of it. But obviously, as you've stated, like the world of entities that serve inference in some form or another is much greater than these three companies that we talk about.
Talk to us a little bit more about what the actual world of inference provision looks like outside of the big household AI name.
So the ones you mentioned, they mostly are closed source models as we call it, right, but they do have some open source versions, but they're famous for their closed source models. So we actually track three hundred open source open weights closed source models globally. Upon pricing and consumption point of view, it is really interesting if we have actually, you know, we haven't really formally launched token in disease. You can currently look at Bloomberg and it's on Bloomberg.
What's interesting is people are depends. It's all based on your choices. Right now. The price actually doubled file indusees from now from December first last year. It's like two dollars twenty one dollar per million token. It's a mixture of input up and token prices reach weighted by consumption by buscut models. It's not here. This is a GPU unfortunately, which.
Are just well, since we have this specific chart up right now, what is the y access in this church?
So you look yet that the per GPO power rental rate on demand for three chips. The top one the yellow line is B two hundred new cloud on demand per flower. So it's a mouthful. The line the air line is interesting, right. So every new chip we came out based on historical data one hundred usually came out to be high and then comes down as more supply you know, came life, and then price came down and then stabilizes. So that's the trend we have absorbed for
A one hundred and for H one hundred. So when B two country came out, we pollished the data last year at Bloomberg this early this year, the price was high and it came down, which is kind of what I expected. But the slope was less steep than I expected. I was like, hm, that's interesting, the slope wasn't as steep, and I'm quickly observed the price just came up and now it's higher than the initial opened whatever you call that, right launch prices. That shows your demands apply craft in
a different stage than whatever stage we had before. So the A one the red line is H one hundred neil cloud on demand per jubile power rate. So you can see the price came down last year a little bit sort about the skill so you don't see much, but came down and came back up quite a bit. I think the last three months came up to like eight percent for AGE one hundred. The one hundred is older, oldest chips among the three, right, they're pretty you know,
put don't you calm audit? At this point the price came down, they stabilized, but the price came up about ten to fifteen percent for the past three months. Remember the A one hundred right there, they're not the latest and greatest at all. So this also tells you to supply deman croft shifting.
Oh yeah, actually that reminds me, could you talk to us because you're doing refurbishment of chips as well, right, like, which seems like challenging in many ways and kind of reminds me a lot about like the sort of carbona model of compute or something like that. How are you actually doing this? Like how does that business work?
So this is cool in two different things. One is for people come to conpect change saying that, hey, I want to you know, it's Union Cloud provider. Right, if you get a piece of land, you again a g CO location, Great, congratulations? Then your option is number one. Should I get the latest and greatest, the B three hundred, the gbs the roubmen with a few months or do you want to get refrabit trips and turn out maybe sooner? Right? Then,
to you, it's become our I calculation for the most part. Right, what's you expected, you know, future revenue generation? What's your residual VALUD calculation? How much you can purchase by right, it's actually pretty simple cash flow based in our I calculation.
So the way we approach residual value and refurb trans transaction is you know, based on our I you know this is your potential break even look at h one hundred, right, obviously you're not going to charge its high speed two hundred, but your cost base is also lower, so you can do the future. You're assume a few years of four
country's signing three years this kind of cash flowback. That's your misgivell now, right, So we do that calculation with people so they understand, Hey, what was the value supposed to generate? And it was the treating the market prices for refurbished or used GPUA and you have to test you to make sure things works, and there's out of nuances to that. But we helped go to the understanding of the whole research of value and that's why the whole bubble thinking about but go ahead, what.
Month was it last year when like everyone got really obsessed with like the life span of chips. Remember, like tweeted something about He's like, oh the lifespan, They're like right, and everyone was free, spent like three weeks free and then moved on from that conversation, right, like that was what do we know about chip life spans? Are there misconceptions out there about the how long yes, these can get be productive.
I'm going in to view a few times, but I don't. I mean, I'm not important. Still, I'm more important, but back then that even less relevant. I was telling reporters. I was like, look, I don't know what did that you're looking at based on my I actually have blocks.
So my website, which is completely can to search for it last year because of that conversation, I want you currect later on the second year h one hundred residual value of reseal value for refribit chips about eighty five cents one dollar, So a year later you can sell eighty five to one dollar. That's pretty good. I would say the thirty is eighty four cents one dollar, and I think my cards appreciate win more than that, right, And I enjoined my car forboard every ten ten years,
So it's I had a data. But again I'm not going to argue it against narrative, which is so.
But there was there's a fairly sived a decent drop from year one to year two. That's what after that. You see your general level.
That's November December analysis. Right now, it's a little different. I haven't refreshed to study but our code is there. If you're my data client, you can around my cold you get a number right away. Another thing I want to point out is l forties. They're like you know the ogs right at that time, the people still use them. They charge you hypercure charge your forties as p JB per hour. So you know, I don't know about two years where's the net number coming from? But I will
I will do that trade every single day. You sell you're two years old at ten cents a dollars, I will buy it.
¶ Evaluating the AI Bubble Question
There's a sort of big question looming in the background of a lot of these discussions, which is the B question. I guess whether or not we're in an AI bubble right, And you sort of touched on it earlier. You have all this granular data on how people are actually using compute, GPU prices, all of that. What what's your take on the big question?
So as an index provider, I cannot give any full of guidance, no blamer, nor do I know, right being fairness, I know what do I know? So the way I look at is we had defined a bubble right, so you look at style bubble right and the nest that showed up to two hundred percent and came back down eighty four percent whatever back. Then that's it's a bubble. Right. The way I look at bubble is can is your valiation? Can your future cash flows support today's valuation of yours? Right?
So then I'm not talking about like opening and everyone else valuation. I'm not ob busy. I don't. I don't. I don't understand that process. The way I look at GPUs the machines, it's actually pretty simple. Look at a future cash flow of your forward contracts and then you discounted back. Can you get money back for the price you pay? Right, It's actually pretty straightforward for the machine level. Right, But to your point, right, you can say, hey, what
happened if demand jobbed? No one gonna use your whatever things you have. But remember the four contract is a signed contract. If you have that, you can't know obviously if you have things, the biggest concern is people have over built. You've overbuilt, then by theory, then all your prices will calmed down because it over supply of the market. Right, So then you talk about supply demain equilibrium. How do we know about future demand of GPUs? Right? I don't
know that everyone's guess is better than mine. Probably the way I look at it is not. It's not that easy to bring any GP online, right you can use you hear all those side big deals in the bill, twenty five million dollars invested, but they don't translate to immediate GPU availability right in the servers which you have to be waylisted if you buy brand new stuff co location, you need optic fiber, so there's a lot of unfortunately STAR has to be aligned.
Wait, but people can default on contracts, right, so even if you have a long term contract signed like that could not work out. Could you envision like credit default swaps or something in the compute market like so.
That happens in every other markets, right, aman market if you do OTC trade, you have the rice some monual defon you. Doesn't matter who they are, right, So there's vary There's a lot of mechanism to hatch that. The things you cannot hatch is GPU cost write the price you entered. So that's something exactly will seemmy futures for you can have a transparency the liquidity and then the easiness of treating it out and had your position.
Carmen Lee, thank you so much for joining us.
That was our conversation with Carmen Lee of Compute Exchange and Silicon Data, recorded live at our New York show. I'm Tracy Alloway. You can follow me at Tracy Alloway.
And I'm Joe Wisenthal. You can follow me at the Stalwart. Follow our guest Carmen Lee at Carmen Lee. Follow our producers Carmen Rodriguez at Carmen armand dash Ol Bennett at Dashbot, kil Brooks at Kile Brooks and Kevin Lozano at Kevin Lloyd Lozano. And from our odd Laws content, go to Bloomberg dot com slash odd Lads, where the daily newsletter and all of our episodes, and you could shout up all these topics twenty four seven in our discord Discord dot gg slash odd Lots And.
If you enjoy odd Lots, if you like it when we do these live shows and talk about the birth of a new market, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely id free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.