An MIT Professor Explains His Original Theory For How Markets Really Work

was kind of like their political science department. But I focused on international relations. Okay, I swear this is a complete coincidence, but I also studied international relations. Really, it's funny that, like, of all this time I've known you, this has never come up. No seriously, like for people like who think who are listening, I think maybe we're faking this or something. It's actually I don't. I genuinely

did not know that. About now people are thinking that we just never talked to each other outside of this podcast. We just we just only talk markets, no personal stuff. The reason I was bringing it up was because I was trying to think of a parallel with what we're going to speak about in just a few minutes. Um, And I was thinking, you know, in international relations, there are these two dominant theories that govern how you think

about the world. Their realism and liberalism. Do you remember that. I'm glad you didn't ask me to name them. I would have remembered realism and I would have blanked on the other one. But yes, that sounds right. Okay. So realism is this theory that states and governments and people are essentially self interested and you know, everyone's out to get each other, and liberalism is, oh, actually we can all get along and there's scope for cooperation to diametrically

opposed theories that completely govern that particular study. Um. Now, the reason I'm bringing it up, it's because we are going to talk about a similar parallel in economics. Can you guess what it is? Uh? Why don't you just tell me? I mean, I think, I think I know where it's going. But I really like the way you're taking this. So alright, So it's the efficient market hypothesis

versus behavioral economics. Yes, okay, so most people probably know this, but the efficient market hypothesis basically says that you can't beat the market, that it's a perfect reflection of the information currently out there, in a perfect reflection of the

price that you should be paying for that information. Meanwhile, behavioral economics basically suggests that human beings can be irrational and we can get stuff wrong, and that means that markets also can be irrational and can get stuff wrong. So to pretty much diametrically opposed schools of thought. So are we going to find out which one is correct today? No, We're actually going to talk to someone who thinks they've found a middle path between those two seemingly opposed schools

of thought. All right, I'm intrigued to who are we talking to and what's their theory? Okay, I'm really excited to bring on Andrew Low. He's an economist, he's a long time m I T. Professor, and he's written well, he's written several books, but most recently he has written a book on exactly this topic. Andrew, thank you so much for joining us. It's a pleasure, thanks for having me. So,

just going back to the efficient market hypothesis. I gave a little snapshot of it, but maybe you could describe it a little bit more and also perhaps explain how it came to be a fundamental tenet of modern financial uh theory, when everyone and seems to beat up on it nowadays. Sure, well, you know, it's a really interesting idea and it's the brainchild of two economists. Gene Fama at the University of Chicago coined the term and came up with the basic idea that in an efficient market,

prices fully reflect all available information. And so that's the case, then you really can't beat the markets by using information because it's already in the price and M. Paul Samuelson was the other economist who contributed to this theory, and his paper was titled proof that properly anticipated prices fluctuate randomly, which is a very fancy way of saying that once you incorporate all available information into prices, you don't know where it's going to go, so you can't predict future

prices based upon where it is today. It seems to me the efficient market hypothesis has come under a lot of criticism in recent years. We've seen Nobel Prize winners who have won for their work and sort of talking about this more. The behavioral approach, which is very as Tracy explained at the beginning, is sort of this opposite view, But it still seems for all the criticism that efficient markets has come under, it's still pretty hard to beat the market. Like, it still seems like more or less

it's a pretty difficult task. Well, that was exactly the conundrum that I was trying to figure out when trying to sort through this particular theory versus all of the various different critiques. The efficient market hypothesis actually works pretty well.

It is really hard to beat the market, and prices do actually reflect a lot of information that's out there, and so it really it's hard to reconcile the basic ideas about efficient markets with all of the psychological and behavioral anomalies that people like conoman diversity failure, and UH and others have come up with to try to counteract these various different ideas of efficiency. So walk us through how you tackle that problem then, and the theory that

you came up with you call it adaptive markets. Right. The basic idea is that there are elements of both of these schools of thought that work well, but neither is the complete picture. You really need both of them. They're both important aspects of the same phenomenon, and the idea behind adaptive markets is fairly straightforward. It basically says that investors are highly competitive and adaptive, and therefore it is tough to beat the market because lots of other

people are trying to do that. But it's not impossible because every once in a while, markets aren't driven just by logic and analysis, but they're also driven by human emotion. So, for example, when the stock market goes down by ten or a lot of people are going to start heading

for the exits. They're going to start unwinding their portfolios, and that kind of a herd mentality can actually lead two prices that don't fully reflect the information that's available at that point in time, So, in other words, it reflects emotion as opposed to fundamental valuations. The efficient market hypothesis is a great way to explain market dynamics when people are acting logically, but every once in a while we freak out, and the freakout factor is where the

behavioral economists have their day. Both of these are important aspects of market dynamics, but they don't always operate at the same time, and it's really trying to understand which part of these different phases are relevant at a given point in time that the adaptive markets is focused on. So I think that like anyone who looks at markets can appreciate that there are times when sort of pure emotion and animal spirits really take over, whether it's a panic,

whether it's in the stage of a bubble. But one of the things that we've talked about a lot on this podcast, as said, factors that even if you know something as a bubble, or even if you know something's a panic, it's really hard to know what stage you're in and whether you're near a bottom or whether you're you're near a top. So my question is, if you know you know, you sort of thread this middle ground where sometimes behavioral take takes over. Sometimes markets are based

on pure information. Does your theory help one get any closer to actually, you know, maybe beating the market. Well, it does, and I argue that those who do beat the market today are using some form of this theory. For example, hedge fund managers understand instinctively that market dynamics change as a function of the flora and fauna of the market ecology. In other words, they look at who are the participants in any given market at a point in time and they feel the the market dynamics as

a function of those various different participants. It really is trying to understand markets from a more of a biological perspective than a physical perspective. And uh, I think that that kind of approach will requires us to collect very different kinds of data from the ones that we're doing right now, and if we had that data, we can make much better predictions about where the market is going. So what sort of data are you talking about? What would be helpful just knowing who is and who isn't

participating in a particular market. Well, let me start by giving you a different perspective. Imagine if you're an ecologist being asked to study particular ecological niche say, the Amazon rainforest, and let's suppose that you would like to save a

particular species in that ecology. How would you go about it. Well, as an ecologist, you'd probably start by taking an inventory of all of the different species, how they relate to each other, what they eat, who they prey on, what the various different food chain relationships are, what the environment looks like and how it's changing over time. Once you study all of those aspects of the environment and the

flora and the fauna. You can then start get identifying key aspects of that system that require management in order to preserve a given species or in order to highlight

a particular species. If you now take that same analogy and apply it to the financial markets, you'd see that what you want to start with is not just looking at prices, but to understand who the buyers are, who the sellers are, and not just that, but the nature of the buying and the selling pension funds, broker dealers, hedge fund managers, who the investors are, who the ultimate

buyers and sellers aren't, and what motivates them. Once you understand the nature of the flora and fauna of the financial markets, you can then start making predictions like, well, if it turns out that pension funds are going to be indexing and sticking to a particular asset allocation over a period of time, then that means that they're going to be submitting by orders when the market goes down and submitting cell orders when the market goes up in

order to maintain that strategic asset allocation. You'd understand the motivation for the various different species in that marketplace and be able to make better predictions about how they would react to certain kinds of market events. That that's the kind of data that I think we need in order to be able to analyze market dynamics. Now your book

and your theory is called adaptive market. If that data were to be made available, presumably all of that would then be incorporated back into price because people adapt would that then require some sort of further metadata for investors wanting to stay ahead of the trend? Absolutely, In other words, you really have to take into account the impact of behavior on those dynamics. And that's the same thing in

other kind of biological settings. You know, for example, if it turns out that one species begins to grow, that growth is going to mean that it's gonna be going to be plentiful in terms of its numbers, and therefore it's going to require more food. Whatever it preys on is going to end up being selected out, and ultimately that means it's going to have less food per individual, which means that eventually the population is going to decline.

So in other ways, their feedback loops in the system that have to be incorporated in terms of predicting how one species will do relative to another. The case of human beings interacting with each other's more complicated because we can think farther ahead and plan and predict in much more sophisticated ways. So once we see these kinds of changing dynamics, we're going to alter our behavior, and that change in behavior will then have an impact on those dynamics.

So the system tends to be more complicated, but nonetheless it is a system that can be modeled, and with the right kinds of mathematics and statistics, we can actually do a better job of modeling that system than using kind of static physical laws that we're trying to apply right now to market dynamics. So this is what I'm really curious about, because one of the attractions of the efficient market hypothesis is its relative simplicity as a model.

What you're saying definitely makes sense, but I can only imagine that, you know, identifying and figuring out how a complete ecosystem of a particular market works. Are you actually simplifying anything there? How useful is it as an actual model? Well, all I can say is what Albert Einstein said when he was accused of developing theories that were so complicated and by the way. I'm no Albert Einstein, so I'm

not comparing myself to the great physicist. But when Einstein was criticized for the complexity of his special theory of relativity, he responded that a theory should be as simple as possible and no simpler. And I think that the financial theories that we're using are actually simpler than they should be. So there's no doubt that the efficient markets hypothesis cuts through a lot of really complicated and unnecessarily involved types

of theories that really don't make any sense. And so that's one of the reasons why Fama, Samuelson and others u had such an impact on both academia and industry. But what we're seeing over the course of the last

couple of decades is much more complicated financial dynamics. It's not the case anymore that a buy and hold strategy of a sixty forty portfolio is good enough for retirement, because well, we see markets going up and down in some very dramatic ways over short periods of time, and if we ignore those dynamics, we could actually get into a fair bit of trouble, especially those of us who are thinking about retiring within the next ten or twenty

years versus thirty or forty years. So horizon matters, the nature of the buyers and sellers matter, the fact that we have an inner, nationally integrated financial system that's different than it was thirty or forty years ago. So I think that we do need to have more complex theories to match the complexity of the financial system as it is today. But it doesn't mean that that we can't

simplify that kind of complexity. In other words, the theory of evolution is a great simplification of what happens in nature, and it is more complicated than the earlier stories about how we evolve and how we change, but I think that it does capture a very important set of of

differences from those earlier theories. So what I'm hoping is that the adaptive market hypothesis, while it is somewhat more complicated because it contains both human behavior as well as efficient markets as uh sub subsets or subcases, it nevertheless provides a unifying framework that allows both of those theories to live happily under one roof. So I'd love to spin it forward and talk about this market today. Because there are all sorts of interesting debates going on right now.

People say, is there a bubble going on? Has the Federal Reserve created some sort of unusual stability? What explains the lack of market volatility despite seeming you headlines that are extraordinary when you look at this current market from the sort of ecological standpoint that you describe, what are the what are some interesting things that you're seeing or

that you're just sort of exploring in today's flora and fauna. Well, it's interesting that you mentioned those various different aspects of what's going on in the financial system, because they're actually quite closely related, but in ways that I don't think you would have been able to see if you're focusing on markets from the efficiency perspective. So take the example of the Fed. Well, we know that the FED engaged in some very significant quantitative easing in the aftermath of

the financial crisis. Now why is that important? Well, quantity It of easing is a direct way of trying to stabilize markets and increase employment by taking on certain kinds of assets and managing the liquidity of the Federal reserve system. That involved reducing interest rates to a certain level and encouraging investors to put money in riskier assets. So that's

what we've seen. We've seen in a low yield environment, investors have flocked to a variety of risky investments, but the vast majority of the funds have gone into stable, passive index products, and that in turn has actually caused equity prices to rise over the course of the last decade. And that increase in equity prices, particularly in passive vehicles, definitely contributes to the fact that we have lower volatility

today than we had in probably twenty years. That decrease in average volatility, in turn has caused as investors to put more money in equities because of risk parity strategies and other volatility linked investments. So the action of the FED which was in response to this financial crisis, and it was an emotional reaction in a way, because one could argue that prices go up and down all the time, you should just let the chips fall where they may.

But because we care about people who are out of work, we want to make sure that financial stability is a high priority among regulators and policymakers. So that kind of reaction has repercussions that have an effect on market dynamics. And we're working through those effects today. So when people hear the word adaptation, or at least when I hear the word adaptation, I also, you know, I think a little bit about resiliency and you know, again the ability

to adapt to new situations. So when you look at the market nowadays, lots of people are thinking about valuations being sky high, the possibility of things may be beginning to pop as central banks withdraw their quidity. What's what's the fragility that you see in the ecosystem, What's the thing that could topple over the dynamic that we've been seeing for the past five or six years. Well that's a great point, because fragility is something that biologists and

particularly ecologists study all the time. And one of the things that they tell us about fragility is that we need to have a certain amount of biodiversity in order to create a more robust ecology. The basic idea being that certain species can get wiped out because of an environmental change, but if we have a variety of different species, the likelihood that one or two of them will be able to survive will allow the ecology to maintain some

semblance of its prior existence. Even before that big evolutionary shock, we don't have that same kind of resiliency concept in economics. I think that certain economists, over the course of the last several decades have tried to focus on that by looking at things like concentration in the industries and various different types of industries that are starting and those that

are declining. But the idea of measuring resiliency in the economy is really pretty far behind what the biologists are doing. So from my point of view, I think that resiliency is really a key issue, and that's one of the reasons why I focus in some of my research on hedge funds. The hedge fund industry is actually a source of all sorts of new species. If you think about, you know, various different kinds of investment vehicles that are

now widely available. A lot of those investment vehicles first began in the hedge fund industry, and so it's important if you want to maintain resiliency to have a vibrant hedge fund sector where all sorts of new ideas can get tried out, and the ones that work well will progress and grow, and the ones that don't will basically get wiped out. You want to have that kind of turnover in ideas and and financial products and services so that we don't ever get into a situation where we're

getting locked into something. The one concern that I have about resiliency right now is that we have a huge amount of assets flowing into passive index strategies, and obviously that's been a very important source of investment return for a large majority of investors who don't have the skills to manage their portfolio actively. So active versus passive is

a debate that I think has long been settled. Passive is definitely here to stay, and it's an incredibly important component of the flora and fauna of the investors that are looking for opportunities. The problem is that if we now are all investing in these passive vehicles, what happens when there's a stock market correction that inevitably there will be, and we see these passive investments under performing, Well, we're gonna get a massive exit from that particular set of vehicles.

And like any kind of a situation where you've got a crowded trade and a massive unwinding, you can see a much bigger drop in these market levels. So crashes are now more likely. In fact, we see flash crashes happening all the time, which is a technological example of these kinds of phenomenon happening in very very short term. But I think that this is a concern that I have about resiliency. We're creating opportunities for financial panics that didn't exist ten years ago or didn't exist to the

same degree. So we just need to be wary of that and be prepared for those kinds of shocks. Very bleak answer, but obviously, you know, it's definitely capturing an anxiety that I think a lot of people feel right now about the markets. We've talked about this a lot, that sort of where is this sort of endless boom and E T f S and passive actually go. But you know, there's debate about how big passive could get.

There's the study of ecology. Give us any clues into when tipping points could happen or anything like that, or do we just sort of have do you know it'll just happen one day and it will be all over Well, I think it does give us a clue. Rather, it gives us a way of trying to understand where those tipping points might arise, and once again, the way to do that is to measure the biomass of the various different species and ask what they're driven by and ultimately

what will cause them to change their direction. I think that in the case of passive investing, it's pretty clear that it offers tremendous benefits to a large number of investors. So we're not going to see any kind of a decline unless and until there are some kind of widespread market decline. If the stock market goes down by ten or that might be enough to cause a retreat into fixed income assets or cash for a period of time.

And so I think that's really the kind of tipping point for passive investments because right now, given the low yield environment, investors are really continuing to pour money into that sector. But eventually nothing lasts forever. We're going to

see reversals in every kind of asset class. And in this case, if we take a look at the nature of the investors who are going into the market versus those who are willing to take money out of the market, will get a better idea of when that kind of a tipping point might be and and what kind of triggers might cause that tipping point to happen. I'm trying to think if passive funds are the eight hundred pound

guerrillas or the excitable antelopes in the market ecosystem. All right, Andrew low, m i T Professor and author of Adaptive Markets, Financial Evolution at the Speed of Thought, Thank you so much for joining us. Thank you, it's been a pleasure. So, Joe, what did you think of the you know, the ecosystem analogy was strong in that conversation. Yeah, I really like that conversation, and I really like that framework for thinking

about it. You know, it's funny thinking about what we do all day, writing about markets and what people think is going to go up or what people think are good investments. Are not even a pure emage world. We'd kind of have to admit that our jobs are sort of stupid in a way, like why even bother writing about this? Why bother saying? Why bother? Why bother bringing people's opinions if there's no way for anyone to just

sort of get a pure edge. And what I like about Andrew's view is the idea that it's really tough, but it's not impossible and that if you explore the right facet there is value in sort of in at least trying. I agree all the I'm not sure. I

like the notion that our jobs are meaningless. Well. One thing I really liked about his framing, though, is the emphasis on market structure, because this is something that you know, you and I bang on and on about, but in order to understand the market, you really need to understand the structure of it and the various players and the motivation at play. And I really think that's the point

he's making about his theory. That said, you know, I can imagine that that does get quite complex when you're an economist trying to publish a you know, paper for instance. That's that's a lot to model totally. But this idea that you know, there's sort of two different ways of thinking about the market. So one is you might have a stock and you might look at its income statement and it's balance sheet and sort of what we're talking

about with Ozwa Dalmadaran recently. But then the other aspect is this where it's like you look at the overall market and you try to figure out who the gazelles are and who the hyenas are, and who the eight hundred pound gorillas are and figure out, Okay, what is the motivation of each of these And I think that also is a very interesting way of thinking about what's what in the market. Yeah, agreed, I really want to go to the zoo. Now I have this urge to

go to the zoo. Okay, let's leave it there. That was another episode of the Odd Thoughts podcast. I'm Tracy Alloway. You can follow me on Twitter at Tracy Alloway. And I'm Joel Wisnthal. You can follow me on Twitter at the Stalwart. And you can follow Andrew Low on Twitter at Andrew w. Low and follow our producer Sarah Patterson at Sarah pet With two Teas. Thanks for listening.