Bloomberg Audio Studios, Podcasts, radio News.
Hello and welcome to another episode of the Odd Lots podcast.
I'm Joe Wisenthal and I'm Tracy Alway.
Tracy. We did another one of our live shows, this time our biggest show ever, biggest show ever.
It was absolutely amazing. We did it at City Winery in New York. I think we had over three hundred people.
Yeah, three hundred and fifty people were there.
Yeah.
And the crazy thing is, I think it was our sort of first themed show, and we didn't really plan it that way, but it just worked out right.
I guess like it themed an anti theme at the same time, because we're in this moment in which everything is just like AI markets, markets, AI, et cetera. But you know, there's all kinds of new things to trade, and people are fascinated by the trade itself, pro fascinated by the way the technology development is affecting the trade. So we really wanted to do a kind of future of trading show, which is a very broad thing. But it did sort of come out that way.
Yeah, it really did. And you finally fulfilled your longtime dream of doing two part episodes with our guests. So our first speaker of the evening was actually someone who's been on the show before.
That's right, So we had him on the show last year, and we had him on our live show. Listen to our episode with Ian Dunning, he is the head of AI at Hudson River Trading. Talked about all things implementing AI, GPUs, all that stuff within the context of a trading shop.
Take a listen, Joe, this is.
Your dream, right, you finally got to do a two parts.
This is the thing I always think about, which is that after every episode we do, I'm like, Oh, there's a question I wish I had asked. So we had Yan on sometime last.
Year, so the last round was easy, this one will be right.
Worried about that, well, I was going to start first before we you know, talk about what you do, et cetera. So here's the question I wish I had asked last time. So, Hudson River Trading Shop, you're involved in the AI stuff, could you theoretically do what high Flyer did and launch an LLM with this tech stack that you have and launch a deep seat competitor?
I think so. I think we're good at training models. We have a lot of compute and people are good at doing the cycle of research which is required to catch up to the sort of frontier. However, I guess reaching the frontier is clearly a very daunting task. So maybe it's with some effort deep seek, but beyond that, it's not a claim I'd be willing to make. It's a hugely capital intensive task, clearly.
And do you ever, like, do people ever chat about that?
It's we could we think about it?
You know?
I mean, I think perhaps we missed our moment to do so. It is so many open models now coming out from the US as well as like China, that there's like a huge array of them. It's kind of an interesting shift from that deep Seak moment where it felt like it was the first bolt from the blue. He's a competitive open model. So now I see so many groups releasing them. I don't know what the future of open models is. That they're all kind of a serious step back in. The frontier is progressing so fast.
I don't know how you keep up with that, but many people believe that it's possible. I'm not so sure. I'm one of those people though.
Okay, so, speaking of things moving so fast, my first question is slightly different. I looked up your Twitter feed. Oh no, before you came on the show. Your last tweet before today was and I quote feel this every day, worry. It's some sort of AI induce delirium. But then again, various empirical measures are exponential looking, so feels best to
assume we're hurtling towards some sort of endgame. So first of all, please convince us all live on stage that you are in fact not suffering from AI induce delarium. But secondly, like, what is the endgame that you speak of here?
God, now I saw like a San Francisco pressent I do. I'sist. I've been doing AI stuff since around twenty sixteen, and that stuted at Deep Mind, and it was a bit of a culture shock for me, because there are true believers then, and I was most certainly not a true believer, and I resisted it, and it was kind of a natural skeptic for a long time. But certain empirical measures
of the pace of progress in the outside world. And I also look at our own business, which looks somewhat exponentially the amount of compute I'll have next year versus this year, and this year versus last year. Looks kind of exponentially, and we're doing things today that I didn't really I should have dreamt of. I wish I had that kind of visionary say, I'm a visionary and I
can see the future and I'm building towards it. But no, I'm I think I'm a pragmatic engineering archetype, and so it's been very incremental and I'm like, wow, that happened in a year. So what does this mean? It's some sort of technological convergence, everything going faster all the time. Well, give us any I'm probably, but you give us an.
Example then, because you know, obviously those of us using just the regular models, obviously the improvements and capability from one year to another our mind blowing. But from the perspective of like, okay, the application of AI within the trading context, what is something that you can do in twenty twenty six that and say twenty twenty four you would not have been able to anticipate.
Oh, I think it's one way to think is just like the amount of compute going into both training a model and running a model, and that it's the same technology working across every equity, every future, every crypto market, every option market across the world with a kind of unified approach, and this is something that we're doing. But even more interestingly, I would not claim that we are some unique people who are only the ones who have
really made progress in AI and trading. I think many of our peers are also investing massively and we're all doing it all at the same time. And what does that mean, Like, surely you can't just like keep getting better at predicting markets Furvace. It's got to be some sort of forcing function where you know your your margins go to zero as you keep investing.
What you're saying is you are just getting better and better at being able to predict where a market is going to go further and further out in the.
Time frame basically, And we're not the only ones. So in the end, can there be some highlander type thing like what are we doing? And it's it's this is like my scale, And I guess the other thing I find interesting is, of course the scale that everyone can see with the big labs and what they're doing of computeion. It's like it looks awfully exponential to me. We just had another model released today from Anthropic and the type of spacing between them seems to be compressing. I do
sound hilarious. I sound feverish, and that's why.
It's literally everyone in this room.
Probably I feel everyone must feel the fever to some extent.
Yeah, I never understood the Highlander there can only be one thing because they're already two. They could just coexist, that's right. Anyway, Sorry, I'm just picking apart your analogy. You mentioned a new model release. When a new model gets released, like, what is the first thing you do at Hudson River Trading to evaluate it? And how do you actually compare them to the existing one?
So I mean our prime reuse cases as a trading and definitely kind of just like accelerating your own research. So that can be coding, but it can also be
coming up with experiment ideas, monitoring experiments. We had a sort of a false start with AI I would say sometime last year with the Opus four point zero models, especially from Anthropic, where a cursory examination made us feel like, well, this is the moment we've crossed the dividing line, and we had a very feverish week where we felt the AGI and we left feeling empty because we realized that it was not there and was not able to meaningfully
augment human researchers. And then we had that same feeling again when Opus four point five came out and suddenly it was like, oh, wait, no, this is this is actually what we thought it was going to be six
months ago. So in the most recent model releases, the differences have been more subtle, but we see, I think we have a much better sense of an ever reducing set of errors they make, and so we're kind of looking for those those of mistakes, and we are We spent some time in the past couple of weeks trying to come up with objective measures to index them against humans in the active quant research ideating signals and things quant research used to be, as we talked about a
little bit like hand crafting indicators and things, why not ask AI agents to do that and compare them against humans like a little sort of battle. And it's they're like, I don't know, intern level AI. Perhaps the thing is like, what do I think it'll be in a year? And I would not want to make a bold claim, but it will still be.
It'll be wild. So when we think about investing in general, even within sort of like classical quant trade, and going back decades, there is often it might be quant but
there's some intuition behind it. Right, Cheap stocks tend to do better, and we don't actually totally have agreement why they did for a while, but people aren't necessarily surprised by that fact, right are we at the point where it's like why even bother coming up with the human intuitive story and you just skip the part of giving an explanation that's owns logical to a person and it's
just basically pure like rigorous back testing. And then it's like, look, here is something that seems to work, and we've back tested it a million different ways and it seems to work, and we don't even bother coming up with a story for why, but we're going to trade it.
I feel like we're in that world today. It's seat of post post post capitalism. When I see IPOs discussed for this coming summer, at the valuations they are, I'm like, what is a fundamental? Like what is anything? It feels like markets are just the cynical thick is everything is gambling, and so everything is some sort of like gambling market,
including public markets. But the joke is flows, it's buying and selling, and it's just it's worth what it's worth, and it's detached and more biased, and sellers price go up, and models are excellent at like pulling that out of data.
But just like let's say, you know the classic example of like a back test, it is like, oh, companies with the ticker symbol it starts with P, they do well on Tuesdays. And it's like, well, look the data says that, but this makes no sense. We're not going to trade that. Could it get to the point where it's like, look, symbols that starts with P, you do well on Tuesdays, And we've run this a bunch of times and it seems to work, so we're gonna put money beyond us.
AI Like I.
Feel like, yes, although it sounds crazy, it sounds like AI delirium when I say it, but I feel like there's some sense that that could be true at some point I can't predict. So at the very short time scale, people accept this already, right Like I can't tell you the price of like a stock in a minute, and no one would really reasonably expect any human to do so, even if they had the uder book and spend all
the time in the world is staring at it. But we accept that neural networks can do this, And then when does that logic break down? Why should it break down at some long time scale. If it's ingesting all the data and has everything and it can keep it all in the context in way human can't, why should I be able to understand it? Yeah, and that is a strange thought, a loss of control. It feels like a loss of control. But it's like, you know, people
save us from math. Maybe humans are actually very bad at math. So it's not surprising AI as much better than humans that these like mass proofs. Humans probably would be pretty bad at markets where thousands of tradable instruments on like very long time scales. We just kind of accepted that we were some people were good at this. Maybe that's a temporary state of affairs.
Well, we talked about this the last time. You were on the idea that the models themselves are not very interpretable, I guess you would say, but you're comfortable with that on a short trading timeframe, which is what you do. And then we started joking about magic models, and magic is a dangerous word to use on this podcast because
people start thinking about magic boxes. But anyway, now that you've been doing this for another six months since we last spoke to you, do you feel like you have better insight into what the models are actually doing and why they're able to succeed on short timeframes.
I do think there are diagnostics we have done where when we can see things that we can understand. It's like looking at some very very complex thing and you can look at one facet of it and be like, this is the fast that I understand, and that gives you some confidence, but it might be illusory because it's a very very complex object, and you can if you're only taking slices through it and to understand aspects of it.
You know, we had this emergent phenomenon we saw where it felt like the model kind of understood meme stocks from first principles like quantum stocks and crypto stocks being kind of adjacent in stock space, and of course from a fundamental perspective that says, no, there's no meaning to it. But we looked at the model and us under a certain lens and it clearly felt like they knew they were connected. There's some other actual companies that I probably
won't name. It feels like it's bad form, but you know Wall Street BET's favorites, I guess, and they were near the cluster too, And this was like just one little window. But there were other slices we tried to take which just didn't make sense to us. But again it's like, who am I to say?
The model says they're in that vicinity of hyperdimensional space.
Yeah. The one thing for us though, is that when we do have this magical model, it is in a lot of safety around it. Because we're doing this higher frequency trading. We're trading positions back and forth. There's a lot of risk checks that are fully automated in things. I don't know how you generalize this logic to long term discretionary trading, where the idea of like risk checking and that kind of layer of defense around it, it's
not so obvious to me how you apply that. We can apply very strict controls around this model because it's a well posed problem. We're not taking giant idiosyncratic risks in like one name for months at a time. We can sleep at night because of this. I don't know how you apply the same thinking to like a fundamental long, short thing where you have to put a trade on and it's for three months, and you're intentionally taking a
very large risk in a very sudden direction. That's what's the risk management story around the AI if you just give up all control to just the magic prediction.
So you said something on the last time we interviewed you, which is very important. First of all, I feel like in the quote AI trade unquote, people are obsessed with like, what's the bottleneck now, right? And because whatever the bottleneck is,
you probably solved for a lot more money. You said the last time we talked to you, the chips themselves are We're not actually a major constraint for you, and that it was more like citing the chips and the powering the chips, the access to electricity, talk about that. What is the state right now? Let's say like I assemble approach a bunch of people from Hudson River Trading. Yeah, I get a bunch of GPUs. Is it then not trivial to find a place to plug those in.
It's definitely hot to find sites and at shortly times. If I went to the market and said I want, you know, six thousand Blackwell GPUs in a box somewhere in North America for delivery in Q four. I'm not sure such an offering exists at any reasonable price. Like if it from maybe someone will give up a lease and I could snag it. But I think if I went to the market and tried to get a.
Quia sorry just to be clear, the chips are available, but not the competity.
I think if I had power, I could get the chips Blackwell chips for delivery this yet, but I do not think I could get the whole solution. And then if you go into twenty twenty seven for the next generation of GPUs, the Ruben GPUs, they at least for the first like stretch, are going to be very much assault out. And so I think that's like a maybe you actually have on a twenty twenty seven delivery. You have more like finding a data center shell by then. But you need you need to be in cute now
for those GPUs if you want them early. So those things are those things are in demand. I'll say that for sure. And one of my greatest failures has been you know, part of my skepticism has been predicting how many GPUs we would need and a long enough horizon
and it's punishing because you're constantly playing catchup. And one of our competitors put out a podcast this weekend and they mentioned somebody along the lines of the fact they had one data center and it was the data center, and that was their data center, and then as they're hungering, hungry for more compute, they had to go out and find it wherever they could. And I would say, we are in exactly the same boat. You just can't be picky.
It's like you've got like a mega watt there, I'll take it, and it could be you know, not in terms that are super favorable to you.
Because we'll say more about that. How are you actually going out and sourcing this stuff, because as you say, it seems to be exceptionally competitive. And at the same time, don't you guys have an insane data center in like Norway or something.
And it's not enough. Yeah, and it's not enough. Yes. We go to the neo clouds, the hyperscalers, everyone, and it's a constant dialogue and they're all in competition with each other. But in some sense there must be some much bigger, shadowy competition going on behind the scenes behind these neoclouds because they are all looking for space and power and I don't know if that's the true scarce resource,
and they're a kind of intermediory layer over it. I don't know what their process is like for saucing it. But yeah, they have come to us and said, this lease opened up. Can you please get back to us by the end of the day. And for commitments on a long term contract, and our contracts are a long term. This is not spot compute. This is like eight thousand GPUs for three years, four years, five years payment? Do you wanna pay half upfront? Do you want to pay
some per year? A lot of different commercial terms, credit risk on both sides. It's complicated stuff.
Tell us more about the counterparty risk. So it's like you come and you say you want capacity and some data center. I'm from Hudson River Trading. Who yeah, well, this is the kind of thing like this crowd. A lot of people know what Hudgson River Trading is, but maybe in San Francisco or whatever, that's not a household name, et cetera. They want to know for sure that you're gonna be good, You're gonna like pay your bills, et cetera.
How do you establish to the data center that you were going to be a reliable I guess tenant.
Yeah, it's a it's a definitely being a dance. It's getting better at this point. I think we've anted it enough deals enough people that I think we have that, But we've had everything from people being like, oh, you've issued bonds, what's the rating on those too? Not wanting us to sell too much of one site because if we take all their power rights and then go bust, they might have a long lead time where they can't
get into a tenant and fill that. And so there's a kind of two potty problems of us where it's like they want customers, but there's presumably a lot of customers, but maybe not as many customers are willing to do the big size and pay more upfront. But you know, we're looking at their CDs is on some of these ones and thinking about how that affects our You know, maybe we should pay you three fifty an hour and take out a CDs for ten cents for our equivalent
of insurance on your heavy leveraged neocloud. You're having a disruption, no names, but you know it's I think is reason to be cagy on both sides because this has all come from nothing like a year ago. We weren't there asking for it, and they didn't exist to sell it, and so the only rock is Nvidio I guess, an extremely well capitalized entity who is not going anywhere, and it's making love GPUs and we have a very positive relationship with them. I think that is also a material factor.
How much optionality do you actually have on GPUs now, Like if you say you want to prioritize latency or throughput, like can you get the chips that you need to specify on one of those things? Or like do you just take what you can get.
Or build your own? Yeah you can? You can. Well, many people I guess now are working on building their own chips for for inference, which is as strictly similar technological problem, and ourselves and many of our peer trading firms have hardware teams to tackle this and you can outsource pots of it process, so it's not as daunting
as it seems. But it's definitely an active, very investigation for us and now clearly everyone because I feel like everyone's talking about their partnership with Broadcom or something like this. And if someone says partnering your broadcom, it's like that they're making an inference chip.
So it's interesting, right because you hear about like Amazon, like them Google tp So we could be in a world in which we hear of like a Hudson River Trading branded chipise.
I don't think we'll sell it, but yeah, but yes, you're right, that is that is definitely the right model. And on the other hand, Jensen Never sleeps and Jensen purchased Grock and you know they have got their new product lineup from the groc acquisition, which is a very compelling product as well. And there are others setups etched comes to mind. So the inference space is a smaller
design space. It's not clear that inhouse solutions will be a necessary thing in the future if physic enough people competing. But on the training side, I mean, what a mote like, there's just a video, it's just and I suppose Google. But you know, if you're using TPUs, you're also kind of entering a very close relationship with Google, some feeling of vandor lockin. It's a complicated thing if you go down that past. But if you're compute hungry, like the
neo labs. I mean, obviously the big labs. You'll take what you can get. I think Anthropic takes TPUs, trainiums and GPUs. You know they need them all.
So maybe we'll create a little bit of con controversy here because later on in a little bit we're going to be speaking with Carmen Lee, the CEU of Compute Exchange, which is this, you know, one of the multiple entities are trying to build financial markets for compute capacity right trade it like oil, so it's like compute futures and stuff that right now, could you see a use for that of financial instrument that's like on some liquid tradable
exchange for H one hundred or whatever, some benchmark of how much it costs run these chips. Could you see that being a useful instrument for you at some point?
It's it's plausible. It highly relates back to my previously stated failure to plan correctly for the future. If in some sense I could lock in a price for some future date for delivery or something of compute, something that is connected to for prices compute in the long term future, I think that could be value to that We could basically hedge our risk that we wait too long to put the order in and it price goes up. I mean, in twenty twenty six, the priceive memory has gone up
so much that we do have concrete specific things. I wish I put that order in a month earlier, so it's a real real thing. Do I believe that there would be a good market with left liquidity for long dated compute futures? That I guess remains to be seen. I don't know what I would do with a short
dated compute future. I do think defining what compute is is pretty hard, and I have no idea what physical delivery would be if that is indeed of interest because you know, because of a long term contract and because of how much work goes into every site, Like when we connect to a neocloud site, we're thinking about how
to connect it back to our other sites. Everyone's got a different networking system, the file system, Like you know, visit of a GPUs, which was all the focus, but this's also like you know, how is data stored of a site or is it stored of itt site at all? Is there an adjacent site that all the hard drives are in and they're all idiosyncratic, and I can't do anything of one hundred and twenty eight GPUs. I need
thousands of GPUs or bust. That's like my lot size, And so it's very hot to see how you could kind of break that down into useful units. But maybe it's just a spot thing and if it's long dated, I don't know, they could be.
We'll learn more.
Yeah, I did get a preview and it is pretty cool, like the actual program where you can select like the type of compute you need from a specific data center that has like literally I think dozens, if not maybe hundreds of parameters at the moment. So maybe we can get a demonstration from Carmen. What is your token spend at the moment? Is it bigger than Joe's?
I have?
I think I what is my average? I think it's on the order of one hundred two hundred dollars a day per employee's on my team. I feel like that's kind of what I've been seeing lately, and some people are more in a thousand a day range, burst bit bursty for that.
Wait, do you like those people because they're supposedly more predictive?
Definitely not trying to encourage that. I mean, some people go through sturges of experimentation slash Ai delarium, which is understandable. And I think we are always trying to understand the people who are using more like are they doing it for something that you even't figured out yet? That's a pretty profound new expense to have. It's it's it's not at the level that concerns us. Well, it didn't exist at all as an expense type. So that's kind of interesting to think about.
Well, I'm curious, like you know, for the consumer models, they talk about how psychophantic they are. Does that happen? It's like, yes, you're close. This is really smart. You're close to cracking the code of the market. Keep pursuing this. They just one more this idea is doing or is it Claude likes to say, this is doing some real work here in this argument.
It's really good.
Do you get that in the engineering content?
I think we do. And it's interesting. We have we decided a new internship for the summer, and uh, in previous internships, we noticed that, you know, it's quite daunting coming into this quent training context. You know, you have no there's not much to like read a book, can't read a tetbook about it. It's useful, So people ask AI, and uh, you know, it would always mention some things of like an unusual frequency that maybe an expert in
netfield wouldn't like focus us on some things. And we noticed in our like winter internship program a lot of sort of very technical quant finance research terms being mentioned a lot by the interns that no full time are used. And it's like the original seed of the mind virus is AI. So there's a little stuff like that. But our token spending is going to go up, but I mean that's almost guaranteed, and yeah, we're getting value out
of it. Maybe not two x productivity, but I talked to someone who said that team's fifty percent more productive. That's pretty good. I mean, you'd definitely pay one hundred dollars a day for that. I just don't understand how people who are token poor could keep up with someone who's token rich. And that's a gain goes to the
acceleration feeling. It's like if you have two people who are sort of equally resourceful and smart, but someone who's basically a co pilot with them, that's giving them a fifty percent boost and all they have to do to get that is essentially spend money. It creates a have have no dynamic that possibly compounds. As you have more success, you make more money, you're more willing to eat now a thousand dollars a day per pres and for token
spend you go even faster. And this feeling again of compounding acceleration, which might be delirium, but you could make an argument for why it could be a real effect instead of more winner. Take all contexts where speed of improvement is like the key thing. There's a story there, I think, or delirium matter well.
I mean, speaking of the haves and have nots. The other big story in AI world is just competition for talent, right, and everyone is sort of chasing the same genius engineer. I guess how are you finding that At.
The moment, it's changed a bit. There's a lot of dynamics going on. There's still a feeling that if you are plucky enough, you can get a VC to fund your idea based on very little. You have the right pedigree, and that's always been true. I guess some sense this is like the YC philosophy. Some sense you go and it's just that some of the numbers and the foamo feeling and is quite shocking. And so that's actually a form of competition, just like why didn't I go create
a stetup. I don't have any ideas or anything. I'm just going to make a startup for the big labs. A question of upside remaining upside. You're at a trillian now, I guess there are two big ones that are a trillion dollar evaluation. Where do you go from there? I think that's affecting people's level of like forward looking optimism for people who are taking offers now and for people
who are at those places and looking to leave. Generally, it's a question of like, well, they've become big tech, They've added people at a vast rate, and the culture has shifted, especially at some of the labs a lot, and to our favor. For a while, it did feel like we're in a very, very fierce competition. And now maybe it's now it's maybe more even playing field. But I don't know. I talk to a lot of undergrads and they don't feel great about the future. They feel very worried.
Basically, what I was gonna just went by way too fast. But like you mentioned already, the models are like okay, junior level, Yeah, what does talent look like at this point? And what are like I've seen some of the anthropic interview questions and it's like designing some GPU kernel or like optimizing the configuration of GPS within the data center? Did what do you want someone to bring to the table at this point?
I mean, I think the first thing is just trying to embrace an open book philosophy, like let the interviews be done with the aid of AI is something we're trying to aspire to do because it's just at some point you become it becomes unrealistic to pretend anyone would work without that. One of the big things in quant is being like this is like archetype of like the math theorist or the string theorist or something and they go in to Long Island somewhere and they come out
with alpha. But you know, like our experience has been a little bit more mixed because it's like, if you can't implement your ideas, how do you how does that happen? Exactly? Well, now Claude can presumably implement the ideas, So trying to embrace that maybe we do accept more theorists, more dreamers, people who can come up with ideas, trusting that the implementation work can be done by AI. So I think
that's our shift. But I've been joking. It's like the word cell versus shape rotator type, Like I feel like the error of the word cell may be a bonus, like if I did prompt. Yeah, I mean prompt engineering is kind of a boomer term at this point, but there is something to be said for like describing what you want clearly and without confounding factors, and that is a skill that can be learned, and that's not evenly distributed in the population. And I would argue that as
shot up in value simply because of AI. So I like to think of myself as one of these people though, So that could be the Delarium'm talking.
I don't know, all right, Ian Dunning, we could talk for two more hours. Thank you so much for joining us at.
That was our conversation with Ian Dunning of Hudson River Trading, recorded live at our New York show. I'm Tracy Alloway. You can follow me at Tracy Alloway.
And I'm Joe Wisenthal. You can follow me at the Stalwart. Follow Ian at Ian Dunning. Follow our producers Carmen Rodriguez at Carmen Arman, Dashil Bennett at Dashbod, Calebrooks at Kalebrooks and Kevin Lozano at Kevin Lloyd Lozano. And for more odd Laws content, go to Bloomberg dot com slash odd Lots. We have a daily newsletter and all of our episodes, and you can chat about all these topics twenty four to seven in our discord Discord dot gig slash odd Lots.
And if you enjoy odd Lots, if you like it when we do these live shows and talk about how trading firms are actually using AI, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening. In behead