Annie Duke Discusses Decision-Making in Poker and Life

about thought process. It's all about not looking at outcomes, but thinking about how you think about what you're doing, whether this is business or finance or investing or what have you. Uh, there is a run of cognitive issues and there is a run of miss focus on what we do, how we do it, what we don't know. But should our own blind spots, our own cog the neveras that really applies to everything. This isn't just a tele poker book. In fact, I would say poker is

really a minor part of the book. Um it's the leaping off point for discussing human cognition, decision making theory and how we think about the worlds or should think about the world probabilistically, and we very often don't determined. Poker is called resulting looking at outcomes as opposed to process.

Anybody who manages other people's money for living, anybody who engages in behavior where there's a decent amount of risk and uncertainty, should really uh, not only listen to this conversation, which I found fascinating, but get the book and plow through it. You will learn so much. It's it's absolutely fascinating. So, with no further ado, my conversation with Annie Duke. I'm Barry Rihults. You're listening to Masters in Business on Bloomberg Radio.

My special guest today is Annie Duke. At one point she was the winningest female poker player in history. She won the World Series of Poker in two thousand and four, and she is the author of a fascinating new book, Thinking in Bets, Making Smarter decisions when you don't have all the facts. Annie Duke, Welcome to Bloomberg. Thanks for having me so. I was struck by some of your definitions in the book and how much they reminded me

of of investing. And my definition of investing is deploying capital on the basis of limited information about an unknowable future. That sounds a lot like the way you described playing poker, it's almost exactly the way that I described playing poker. So, Uh, the kind of loose definition is decision making under conditions of uncertainty over time. Uh. So that would be a relatively loose definition of poker where you're just as you said,

you're deploying capital based on limited information. That's one source of uncertainty, um about some sort of uncertain future. Uh. That would be luck um intervening, which is the other source of uncertainty. So when we talk about decision making under conditions of uncertainty, those are the two sources which are very nicely put into your definition of investing. And you also spend a lot of time describing the focus on results um and and outcomes rather than the process

that led us do those results. Tell us a little bit about what led you to that analysis and why so many people look at a bad result and think immediately it's a bad process when that may not be the case. Well, I think that it's really hard. So we have this very uncertain relationship between uh, decision quality and outcome quality. UM. So for example, in poker, I can have the very best hand and I can still lose.

I could get dealt asis, and you could have a seven in a two and the turn of the cards make it so that you win the hand, or vice versa. I could have the seven in two and make terrible decisions in playing that hand and still win. And that's really really similar to the kind of decisions that we make in life and investing in business. And the problem is that getting to be able to see the process isn't transparent. The thing that we can see is the outcome of the process. We can see did it work

out or did it not work out? Did I win the hand, did I not? Did the you know, did the stock I invested in go up in value or down in value? And that's what we can see. And now working backwards from that into what was the decision process is really hard. It's very opaque and very often, uh, the quality of the decision doesn't reveal itself except over time. So we can see the outcome right then, but very often whether the decision process is good it takes a

lot of time to reveal itself. So what do we do under those conditions of uncertainty? We have this bias, we have this heuristic which is outcome was bad? Okay, that must mean the decision was bad. I'll take that as a signal outcome was good. Okay, decision was good. Um. And the problem is that that's a really poor strategy for learning from your outcomes. It's great if you're playing chess, but it's terrible. If you're playing poker, it's terrible. If

you're investing, it's terrible. If you're running a business, it's terrible. If you're choosing a romantic partner, it's terrible if you're driving. That's that's the thing about it. So that I love the term for this in the book that poker players use. They call it resulting. Yes, So resulting is taking the quality of the result and deciding that that tells you what the quality of the case. That is not the

case at all. So I opened the book actually talking about this Pete Carroll um Super Bowl and the what was it? Four yards to go go first down, second, second down, and only a few seconds left, but enough for multiple plays if the clock is managed right, right, So peak four, that's where the number that's yeah, that's where the number four came from. So yeah, they're on they're on the one yard line there against the Patriots.

Of course, um, and it's literally seconds to go. It's twenty six seconds left and the Seahawks have exactly one time out. And I think that people remember because it's quite famous that Pete Carroll called and running play there. Uh sorry, Pete Carroll called a pass play there, right, so uh while having the best short yardage UM runner in the game in Marshall Crunch as it is, it was Marshaw Lynch. Marshall Lynch. Yeah, he's a musician. There

you go, marsha Unlynch. So so everybody expected him to hand it off to Marsha Lynch. Instead, he called for Russell Wilson to pass. Uh. The past was very famously intercepted. And during the game, actually you can hear Chris Collingsworth call this is I mean, he's just, you know, gobsmacked. This is the worst thing I've ever seen. Uh. And the next day the headlines were equally brutal, and the argument wasn't so much about whether Pete Carroll had made

the worst call in Super Bowl history. It was more like, did you make the worst call in Super Bowl history? Or was it the worst call in football history? Period? So we can agree that this was a terrible result. But I make the argument in the opening of the book that this is a really good example of resulting, which is just assuming that the quality of the of the outcome is telling you about the quality of Pete Carroll's decision making. There were, in facts, some outline voices.

One of the stronger ones was from Benjamin Morris over A, who argued it was pretty statistically sound. How many remind us how many interceptions were in similar circumstances during the season during that season, zero and historically between one and two. So really this was a high probability. Either there's a touchdown or the passes dropped and you managed the and you still have time for two more running place. Well,

I'm gonna put it investing in investing terms. Remember, he has one time out left, so if he calls the running play and Lynch fails to get in, of course the clock is running then, so he has to burn a time out and he'll only have time to run the two running place. Um, there are three possible outcomes from the past play. One is this really really tiny

chance of an interception. The other two is the balls dropped, which happens in basically no time, So the clock stops immediately obviously, which then gives them the two running place, or it's caught for a touchdown and you assume that the Seahawks are going to win. So let's say that it's a practically free option at the same two running place that you would get otherwise. And we all know

from investing that options that are basically free. I mean, you know, one percent isn't a lot, right, It's a basically free option at these same two running plays. So I think if we think about an investment terms, look, we can quibble right about whether in fact that was a good decision or not, but calling it the worst is as in Super Bowl history, when he was essentially exercising and almost free option, that seems a little much.

And I think that we can see how strongly this is coming from this resulting problem when we just do this simple thought experiment. What if he had passed the ball and it was caught in the end zone for a touchdown the Patriots and what little time they had left failed to score and they won the Super Bowl. What do you think the headlines would have looked at It certainly wouldn't be time. I think it would have been something like out Belichick Belichick, Um, you know, wow,

that was so unexpected. That's what got them to the super Bowl in the first place, that kind of decision making. And in fact, we kind of know that's true because we sort of just ran the experiment. We just sort

of saw that. It come to play with the Philly Special where everybody called Peterson brilliant for not just going for the extra three points at the end of the second down, um and actually throwing this weird path to the quarterback Nick Foles in the end zone and he caught it, and people said he's brilliant, he thought Belichick, and I think that what we can agree is whether you think that that was a good decision or not. The quality of the outcome should have nothing to do

with whether the decision quality was good. In reality, it was one try. We flipped the coin one time. So I think that thought experiment is very revealing because you can see how much you as an individual gets sort of pulled around by the outcome because it's we can see it. It's heavy, it's like a gravity. Well, hind signed bias is a powerful thing. We talked a little bit about Pete Carroll's decision. Let's talk about how this

applies to poker. You wrote, Poker players who stand the test of time have a variety of talents, but what they share is the ability to execute in the face of these limitations. Explain that a little bit. Well, I think that the issue that we all have is that we're pretty good at understanding what our goals are, right. We can set those pretty well, you know, I want to be I sort of refer in the book to whatever your e er is, smarter, faster, healthier, richer, happier,

whatever it is, you figure it out. But the problem isn't so much these kind of like, Okay, I want I know what my long term goal is, in the sense like we can see this with New Year's resolutions, right. The problem is that we have all these small, little executional decisions along the way, and that's where we really

get into trouble. If we think about it in terms of say Danny Kanaman's framework from thinking fast and Slow, we could think about the goal setting is happening in system too, in deliberative mind right, and the the actual A lot of these executional decisions are happening in system one, and we know that system one, this more reflective mind is going to be subject to a lot of biases.

One of the main biases that's going to get in the way of executing our goals is that we're supposed to learn from the outcomes we have, but it doesn't feel good, for example, to be wrong, so we process, and in fact, people hate being wrong so much that they very often will will take a safe play where they don't end up looking wrong, rather than taking a risk with a binary outcome. How did that affect How did that thinking effect the way you played? Did you

see that behavior amongst other poker players? Oh? Absolutely? So if you know that, here's kind of the problem is that you know that the people around you are going to act like the Seahawks fans or the pundits after the play, and if you have some sort of spectacularly spectacularly bad outcome, particularly if it's associated with taking a risk, which Pete Carroll certainly, did you know that they're going to come down on you, right, I mean this the

way that the world is going to come down critiquing you. So what ends up happening a lot of times at the table is that when you have these decisions, and remember your cards are faced down, so people can't see right, they can't really see where they're making a good or bad decision. There, they'll very often come down on the side of well, I'm not going to really do the risky thing, which is say maybe try to bluff here. Instead, I'm just going to quietly fold this hand away and

nobody's ever going to know um. And that's just one of the many problems that can come where you're sort of defending yourself against outcomes. The other way that you can defend yourself against outcomes is that when things happen to you where you lose, what's one of the best defenses. Well, we know that one of the sources of uncertainty that we have is luck. Right, that's one of the things

that we can't we I can choose. I can make a decision that's going to have some set of possible futures that's going to occur, say an interception, a drop past catch, whatever it might be. But I can't control which one right. And that's what I can't control. So we know that luck is going to be a really good defense as well. And so what we'll do is we'll have an outcome it's bad, and we'll say well,

I just got unlucky. And the problem with that is that if you got unlucky, there's nothing to be learned from it. And so how are you going to improve your execution of decisions in the future, because you're just offloading that because it just doesn't feel good. It doesn't feel good to say, well, maybe that bad outcome came from the fact that maybe I didn't actually play the hand that well, maybe I made poor decisions in in my investment strategy or in driving in my car down

the street when I got in that accident. You describe that process of self evaluation of was this a good decision or was this luck as one of the more difficult things that you have to learn when you're playing poker. Why is it so different to untangle skill and luck, be it in gambling or or business or professional sports for that matter. Well, again, because we we generally just don't have enough data to do it, and we tend

to uh field outcomes one at a time. We don't wait wait around an aggregate, even aggregate even if we did so. For example, I mean, if we think about choosing a romantic partner, say you know, a spouse, how many tries at that do we have. It's not like were doing a whole lot of data collection there, right, We only date a few people in our lives. The

end is quite small. The sample size is small. Um, It's sort of like most of the decisions we're doing is saying we're going to flip a coin four times and then try to say something about whether the coin is fair or not, and it's just hard and and we can't see. Nobody sort of pulls back the curtain and says to us, you know, this portion of the outcome was due to luck, and this person portion of

the outcome was due to skill. We just know that we sort of live in this noisy system where sometimes we drive through red lights and we get through just fine, and sometimes we go through green lights and we get in an accident. And it's just not perfectly linked together. Like like it is saying a game like chas, if I play poorly, I probably lose if I play. Not a lot of random luck involved in the of poker exactly, but poker and investing there is a lot of random

luck and we can't see it. We need like a big sample size in order to be able to get down to say something about those statistics, and even then we're probably making some assumptions as we're doing our analysis. And that's why it's so hard, because again, all we can see is the outcome. Right, we can make a bad decision and it can converge on a win. We can make a good decision and it can converge on

the same win. So how then do you just take this thing and now try to work backwards and untangle luck from skill when it's very hard to see down into it and we generally don't have a big sample size. So let's talk about that untangling. Undergraduate, you go to Columbia University graduate school, you go to University of Pennsylvania, you almost get a PhD psychology. How far away from were you from from a degree? I needed to defend it, okay,

and that was done. But you you so you know you're practically you go to the ninth inning with the one yard line. Yes, I was on the one and then you moved to Billings, Montana. So two questions, A, what motivated the move? And be how important is psychology to your successful track record in poker? Well, let me say, you know, there's been a lot of moments of luck intervening in my life, which I think have been pretty interesting.

So one of them is the moment that I moved to Montana, which came from what might look like an unlucky event, but I guess that it turned out pretty well for me. Um, which was that right at the end, as I was going out for my UH interviews to become a professor, I had been struggling with some stomach issues the last year in graduate school, and they really came to a head, um right at that time, and I landed in the hospital for two weeks with this very bad stomach problem. So I needed to take some

time off. And during that time off, I discovered that you you, oh, okay, Now I don't have a fellowship. I need money. Um. The man I was married to at the time had a house in Montana and said, let's go. We'll just go to Montana for the summer and you can recuperate there. Um. And during that summer I started playing poker. And that's kind of how that happened. Now, when you say during that summer you started playing poker, I'm going to correct you slightly, because you grew up

in a house old. Your brother is a multiple uh poker champion, and you grew up in a very competitive household where there were card games and other things going on. Uh, that you played competitively within the family. Am I wrong in assuming you were playing poker at home against your future champion brother. Well, not exactly wrong, but also not exactly right. I would say that the poker chips came out in the house maybe once or twice a year, and it was some games like I don't know if

you've ever played past the trash. I think the more common name for my dad called it past the trash. I think the more common name is anaconda that you've played so you're passing like three cards to the right three card. You would never play this in the casino, of course, UM, so we'd play things like that. So mostly actually we're playing um games like hearts, um oh Hell, which is a sort of simplified version of bridge. Uh. I played bridge starting when I was a teenager. Uh,

A lot of like gin um. And the thing about that is, while you don't necessarily have this kind of ongoing betting element that you do in poker, you have a lot of the same elements of uncertainty and luck. So being able to reason around. Those kinds of games certainly helped me, uh with poker, So that was, you know, that was the first, definitely the first moment of luck. UM.

The The interesting thing with how do psychology help? Is that when I first started UM playing, I think that people thought that I studied clinical psychology, which would be like understanding like why are you depressed or why are you anxious? Or those kinds of things, And they thought that would be really helpful because I would have some

sort of analysis of your personality. I suppose that might have been helpful, but I've never taken a class in clinical psychology in my life, so I know no more than from my own going to a therapist. UM We're Actually was really helpful was incognitive psychology. What you're studying is these issues of how do you learn and uncertain systems? How do we process information? What are the biases and

heuristics that get in our way? Now? That was incredibly ful for understanding not only the learning problem that I was about to face, in terms of you know, I'm getting this feedback at the table and these outcomes are coming my way, and how do I actually become better? How do I figure out how to be a better decision maker in this game, but also how might other people bias be expressed at the table in a way that I could strategically use to my advantage. Um, and

that was incredibly helpful. Let's talk a little bit about your actual poker career, couch, because you've obviously want a lot of money, and the World Series of Poker is a big event each year. At what point in your casual poker playing did you reach the understanding, hey, I could I could earn a living this way. So when I was in graduate school, I my and I you know,

I was living on my little fellowship. My brother was already a professional poker player, so he would bring me out to Las Vegas maybe once or twice a year, and I, I mean, I enjoyed watching him play, but when he was in tournaments, you can't actually sit behind somebody, and so you were just sort of standing on the rail, and that's pretty boring. So I told him I was kind of bored. Uh, And he actually gave me a

tiny bit of money. I mean it was like three dollars or something and sent me off to play with a little napkin of hand rankings. Basically, Um, like literally written, Yeah, it was well, I knew that part. It was here the hands are allowed to play right, And it was written in a on a with a keynote crayon, like literally on on a napkin. This is what I had.

And I walked over to the Freemont Casino. Um, you know the poke room I think was next to the Carls Jr. And that that's where I played, so that certainly, you know, that was when I was in academic, so it never occurred to me. I would say that pretty early on. Um, in sitting down at these tables in Montana,

I figured out that I could make a living. But you know, my reference was to what the starting salary of a professor was, which I think at the time I was applying for jobs that were about twenty three dollars a year. So uh, that's you know, it's a low bar for for what you felt the living was going to be. I had this really big advantage, of course,

which is that I had these great mentors. I mean, my brother among them, Eric sidell Um was one who was really really amazing and very generous with his thoughts. And uh that cut a few years off the beginning of my learning curve. I would say I had a big leg up, and I was playing against people who weren't you know, who weren't world class players. They were playing in a in a game where you had about three hundred dollars maybe six hundred, maybe six d playing

with ranchers and farmers and truckers. And yeah, and I have world champions coaching me, so you know, the ringer at the table. I mean, as it turned out, yeah, So, I mean, I think the first month that I played, I remember, I had a little notebook, and I've made that month. Obviously, that doesn't mean that that was my expected value, but it looked pretty good to somebody who was applying for dollar year jobs. That's right. So we're ready. You're up to thirty plus, right, So I was like, whoa,

I'm I'm gonna make it. You know. So from from that point when you realized, hey, I can make some money doing this, how long was it for you realized I'm a world class player, I am a poker expert. What was that timeline? Like? That was never? I retired in two thousand twelve, and I never I don't I would think that even even after you won, you never thought, oh, I'm a world class player, or were you thinking, well, there was a decent amount of random locking this, it

could have not been my skill. So I think that this is actually really important for anything that you do in life in terms of what my mindset about it was, which was that I think that there's a difference between sort of being humble in the face of the game that you're playing and humble in the face of the opponents that you're facing. Yeah, so the interesting thing with poker is that you know it's certainly for a human being.

The more that you play, the more you kind of realize that you have no idea what you're doing at this table. That it's such a complex decision problem, right, I mean, all the cards are faced down, and there is a lot of luck, and it's an incredibly hard learning problem, and you're really just trying to get a little bit better. You're trying to get closer to understanding what the primary line of play is. And it's very different than in chess. And chess you can really go

back and you can deconstruct with great certainty. You know what the possible lines of play were, what your responses might be, and you can really kind of work that game to the end. In that sense, it's more of a mathematical calculation. But isn't recognizing the degree of luck in randomness in outcomes and a certain humble uncertainty of your own skill set that meta cognition. Isn't that a

channel Dunning Krueger effect. Isn't that a wrecking uh an acknowledgment of your own skill that you reach a point of saying, oh, there's this much randomness in it, and I'm aware of it absolutely, and there's this much that I don't know and can't know. And I mean even towards the end of my career. So I retired in two thousand twelve. I think it was two thousand ten. I won the NBC National Heads Up Championship and I think that was five thousand dollars. I think was first place.

I don't really remember. Who can remember each half million dollars. It's drink about poker. It's it's not really about the money. It's about like, how do I make the best decisions to get to where I'm going? So you actually sort of it's actually a necessary factor to separate yourself from the money, because otherwise the money can cause you to why surgeons don't operate on their own family either, exactly like you need a separation, so you actually kind of

don't think about things like money. So I mean, I could look it up for you what I want, But I think that even then, like right before that tournament, my game had made a big change, like I discovered new things about the game. So it's I think that's it's this paradox, which is that in order to get better, you sort of have to recognize your actual lack of

expertise again in relation to the game. Now that's different than me feeling like if I sat down at a poker table that I would be better at this problem than you, which I do. And then also I think that allows you to recognize when you're sitting at a table with someone who's probably better than you at this problem. And in fact, the NBC National Heads Up Championship gives a good example of that. I ended up facing Eric Sidewn my mentor um in the fine and I kind

of and I knew he's going to outthink me. I mean, he's just a much better player than I am. So I injected a lot of luck into that match because I understood that if I have to execute a lot of decisions against him where he has an edge on every single decision, right, I'm going to be in big trouble. But if I could get my money in one time hoping to sort of get close to a fifty fifty, that I would be better off because I could just win that. So he couldn't just sort of chip away

at me with his edge. Your play was, let's go for the random outcome, and I have a much better shot. Yeah, because you're a better decision maker than I am. And so if I let you make a hundred decisions against me, that's a sure loss for me. But if I if we just go for one big decision, you know you can get lucky on the one coin toss. In investing, we call it relative and absolute returns, and you're you're

making it that way. Let's talk a little bit about blind spot bias and you you actually write in the book, as have other people, blind spot bias is greater the smarter you are. Why is that? Well, let me ask you a question. If you had to bring somebody in to argue your case for you, do you want like the smart guy or the not so smart guy to do that. You want the most accomplished, successful Harvard Law school professor who can argue that case as opposed to

a person who's not very good. Okay, so this gives us the reason why being smart makes it worse. Let's think about us as we're all arguing our case. That's the thing. We all have beliefs that we think are true or false. Right, we all make predictions about the future, and we are really good at making a case for

our stuff. And in fact, we kind of know that because we can see when other people are like clearly just like arguing their side and leaving data out and spinning the facts and putting a particular frame on it. So here's kind of the problem when you're really statistically adapt or you're very mentally agile, is that we all want our beliefs to be true. We all want our predictions to be right, and so we will argue in order to make our case. We're kind of our own

best pr agents. And the smarter you are, the better you are at the spin. So if I have to go and say make a statistical case for say gun control lowers crime or gun control increases crime, give me those two cases to make. Depending on what my beliefs are, I can go and find lots and lots of support

for either side. And if I if I believe the first thing that it lowers crime, I'll go find lots and lots of reasons why that might be true, and I'll give you very good arguments, and I'll show you studies that show it, and it will be a compelling case, and I'm going to be pretty good at it, the smarter than I am. Likewise, if I think that gun control increases crime, I'll go just find lots of stuff

for that. And the interesting thing about it is I won't know I'm doing it necessarily, so that confirmation bias is completely on subconscious and you're unaware of Well, there are definitely people who are spinning. I mean, it's not like people aren't sometimes purposely doing that. But I think that the really insidious part of this is that we

generally don't know that we're doing it. And that's where I think that you know, I think that we have this idea that if I tell you about confirmation bias or the larger problem of motivated reasoning, which is reasoning toward a conclusion, that you already believe is true, which would mean both finding confirming evidence but also discrediting evidence

that disagrees with you. Um, if I tell you about this problem and you're a pretty smart person, that you're going to walk away thinking, oh, well, now I know about this problem and I'm really smart, so now I won't do it. But the thing is that that's just not the way the brain works. These processes are very deeply embedded in again system one a reflective mind. Um,

and we can't do that. And that that really, I think gives us this really good clue, which is if you're doing it, I spot it right away, right, I'm like, uh, Barry, like, come on, you're missing this study, or you're clearly spinning this, or you're distorting this fact. If we know that, what we can do is use that to our advantage and bring other people in essentially to watch our bias back. Right.

And if I can get if we can make an agreement, and I can say, look, I want to have this kind of charter with you as my friend that when you spot me in this place where calling call me out for spinning, if you have information that disagrees with me, please offer it to me. I will try my hardest to not be defensive. If you think that I'm being defensive, you'll call me out on that too, And we're going

to really help each other. And then I'll do the same for you and get some people to go do that for yourselves, because we're good at seeing it in other people. We invest so much time and effort and energy creating a model of the universe around us that it's always a challenge when something undercuts that belief system.

In fact, here's a quote from the book, instead of altering our beliefs to fit new information, we do the opposite, alter our interpretation of that information to fit our beliefs. Will well known as cognitive dissonance. Why are we so hell bent on not changing our minds even when confronted facts to the contrary? Well so, I think again, I think Conoman really has a lot of great stuff to say about this. Generally, we want to have a positive

narrative of our lives. Now, surely this kind of goes back to that long term goal versus you know, what are we doing on execution? We can recognize it in the long term, the more accurate representation we have with the objective truth. In other words, the more that we're willing to change our beliefs as we get new information, probably the better our long term return is going to be.

And that means that our decision making more more accurate. Model. Yeah, and then we'll have a better narrative of our lives. But again, remember I said, we're not very good at aggregating, right. We like to do things as it comes in. And when we get outcomes that come in, either information that disagrees with us or a bad outcome or whatever it might be, it's this momentary. It's like, in the moment, it's a hit to ourself narrative because being wrong feel bad.

Finding out that our belief might have need to be altered feels like being wrong. Finding out that a prediction that we thought would come true doesn't come true feels like being wrong. And all of those are hits to our positive self narrative. So what do we do in the moment? We swatted away in whatever way we can. Either we say that something was because of luck, that it wasn't our fault, that we couldn't have known, how

could we have seen that coming? That was out of our control, Or we discredit the information that disagrees with us, you know, we'll say that person is incredible, or that study doesn't have a big enough sample size, or the statistics that they ran on it were bad. When we see things that do agree with us, we hold no critical eye to it whatsoever. It's just like, yeah, that's right.

So you can think about like I think about this, like, if let's say that you're, you know, your big opponent of trend following, and you really think that trend following is a ridiculous investment strategy, what are you gonna do When you see something that's very critical of trend following,

you're going to go, yeah, great, absolutely. And when you see something that suggests it maybe incorporating some trend following ideas into your own investment strategy might be a good idea, and it has all sorts of reasons for that, you're going to literally pick it apart and talk about why the author is incredible and everything that's wrong with it, because otherwise you have to update your own belief and when you update your own belief in in sort of

you downgrade it, right, it doesn't feel good. And I think that what what you need in order to kind of get around this, and this is where like having people to help you really is good, is to have your identity not be driven so much by this idea of right, meaning I have these prior beliefs and I want to I want those things to be right. I just want to reaffirm the things that I already believe and switch to this idea of I want to be accurate.

That accuracy is what feels good. In other words, building the most accurate model of the world, which requires that you view your beliefs as in progress. And once you view your beliefs as in progress, you can't really be right or wrong anymore. They're just provisional until the next upade.

You're just provisional until the next upgrade. And you're gonna help me do that, because when when you see me start to get defensive and start to discredit something that you know disagrees with a belief that I might have had, you're gonna say, hey, don't do you think maybe you're doing this thing and don't remember the belief is in progress and how much would you change your belief now? And you know this is going to help you have

a more accurate view. And this is where the title thinking of bets really comes from because the person who wins in a bet is not the one who affirms their priors. It's the person who has the most accurate

model of the world. And so thinking about uh, you know, decisions as bets, as as you said, you have some limited resource that you're investing based unlimited information on an uncertain future, allows us to view our beliefs more in progress because that just wraps the uncertainty into the whole process and allows us to be more accurate in the

long run. Some of the my favorite writings on probability theory and cognition and our own internal models mentioned something that you mentioned early on in the book, which is, if you want to be better at any probabilistic exercise, be gambling or investing or what have you, you have to be willing to say I don't know or I am uncertain. Explain why that's so important and why so

few people are willing to do that? Sure, So on the why it's so important is I think that top of the list is that it's a more accurate representation of your beliefs and predictions. Predictions by definition, even if you have all of the information, even if everything is known. Predictions have to be uncertain just because, uh, we don't

know how the future is going to turn out. So, uh, the best example I can give of that is if I have a fair coin, I can tell you for sure that it will land heads fifty of the time, that doesn't mean that I know it's going to land heads on the next try. So that prediction must be uncertain, and beliefs are uncertain as well. There's all sorts of things that what did believe when you were twenty that you were sure of that? Now you look back and you say, wow, twenty year old Barry really had that.

You don't have to go back, go back to four or fifty. Yet it seems every five years you look back, at least if you're trying to evolve, and you look back and say, man, I believed a bunch of junk, or I could downgrade some of those beliefs and maybe they weren't nearly as important as I previously thought they were. That's exactly right. So so so that's the thing is that saying I'm not sure, it's just it's just a

better representation in the world. It's just more accurate to what the state of our beliefs is so that that's number one. Uh, and then number two acknowledging on certain Here's the big problem making decisions when you don't have all the facts. We can't control luck, right, luck is okay? You know it's gonna land heads fifty of the time. I really can't control the coin on the next flip. What I can do, though, is get more information. So

how do I do that? I have to make sure that I'm open minded to the information and also let them information hungry. So once you start not acknowledging uncertainty, what does that do? It creates this information hunger in you. Because if I'm certain about a belief, why am I doing research on it? Why am I going and seeking out information that might help that belief? I'm already sure. But if I acknowledge my uncertainty, this is why it's

so important. It makes me information hungry. It makes me want to go find it makes me open up Google. Not only that, but here's the great thing is that when I acknowledge my own uncertainty in the way that I speak, it gets you to start telling me what you know. Because when I say for sure, like let's say that we'll go back to the trend following. Um, let's say that you're somebody who's a real trend following person and I and I just say, I don't know

that about you. And I just say with certainty, well, people who are trend followers, they're just idiots. I can't believe they think that that's a good thing to do. Do you think you're going to share with me any information that you have about why you think trend following is a smart strategy. I like to argue, so perhaps perhaps, But I'm probably shutting you down right because why either you don't want me to think that you're an idiot, or you maybe you just don't want to embarrass me,

and so you don't speak up. Either way, you're probably kind of mad that I said it. But if I say, you know, I've always been a little bit of a trend following skeptic, you know, but I'm not like, I'm just I'm not sure not etched in stone. Now you trend that following enthusiasts, You're now going to start telling me all the things about why you think that it's

it's a great thing. And we're actually going to get into a discussion where both of us are going to benefit from it because you're going to learn from my skepticism. I'm going to learn from your enthusiasm, and we're gonna find we're gonna get closer to what the actual truth of the matter is because I've opened the door wide to say, tell me what you know. And that's the thing, that's what we have control over. We can't control luck.

We can control the quality of our decisions, which means getting more information and refining and calibrating our beliefs, which I can only do through information coming my way. We have been speaking with Annie Duke. She is the author of Thinking in Bets, How to make smarter decisions when you don't have all the information. If you enjoy this conversation, be sure and check out our podcast extras, where we keep the tape rolling and continue discussing all things probabilistic.

You can find that wherever finer podcasts are sold Bloomberg dot com, iTunes, overcast in SoundCloud. We love your comments, feedback and suggestions right to us at m IB podcast at Bloomberg dot net. You can check out my daily column on Bloomberg you dot com or follow me on Twitter at Rid Halts I'm Barry rid Halts. You're listening to Masters in Business on Bloomberg Radio. Welcome to the podcast,

and he thank you so much for doing this. I was taking notes, which I never do or rarely do, because I have some things to share with you, which you're hilarious. You mentioned trend following as a sample. Um, are you familiar with Michael Kobol's book trend Following? Well, I am. I've actually been on his podcast three times, you have. I really enjoyed the conversations with him. He's a fascinating guy in his background. Everything with Vietnam is amazing.

Have you seen the most recent addition? And who wrote the forward to it? No? Isn't you? I love it? How hilarious? That is so hilarious. So I actually picked trend following as an example because I I kind of follow uh financial you know, somewhat, and I see that there seems to be a lot of emotion around it. So I like to pick topics that are emotional because if if I ask your opinion about skin cream, it's

like you're probably okay. But but the whole thing around trend following, it's like people seem to really it's like there, it's like the Jets and the sharks and they're gonna meet have a riot there. There is something to the concept that if you have a systematic approach to investing and it's mechanical and you follow it with tremendous diligence that you know, if you go back to the original UM book, Oh god, I'm doing a blank on it.

Uh that that the UM One of the turtles that were mentioned in Market Wizards is the original book by Jack Schweger, the turtles that Mike Covil talks about all the time. If we could raise traders the way they grow turtles for export out of out of Asia, you know, can we teach them mechanically? And it turns out you more or less can assuming people are willing to do that. Yeah, well,

you know, I think it's interesting. I think that when markets, when there's some width in the market in particular UM, that becomes very easy to to kind of do these kind of mechanical strategies. So the way that I was related to poker, which I always thought was very interesting, was so I used to teach a lot of poker seminars, and in fact, I'm a big proponent of if you

want to get better at anything, try to teach. So, Mike, when I started teaching seminars, there were a whole bunch of things that I was doing at poker that I actually ended up throwing out because I realized that I couldn't actually explain it coherently to another human being such

that they could execute the strategy themselves. Um. But one of the things that that the and these were generally intermediate players would would come up in One of the most common questions they would say to me is don't you hate playing against beginners? Is it don't you hate that because of the potential random outcome? Or I think because there was more it was more unpredictable. Um. And I would say to them, what, No, I love playing

against beginners. And my question to them was always this, if you had to play like a tennis match for your life, like someone's literally going to shoot you in the head and kill you if you don't win, do you want to play against the beginner or an intermediate? And they would say, well, the beginner, of course, and I said, well, it should be the same for poker.

And what I figured out was that the reason why they felt that way is because there's this thing you can do in poker called bluffing, and bluffing is really exciting, right, It's like high out smart, did you I had the worst hand and I made you fold the best hand, and pa, right, I'm great. Well, bluffing can be very hard to do against beginners, and the reason is that beginners kind of don't know enough about the game yet that they understand that your actions should cause them to fold.

So so the takeaways don't bluff against beginners, right, except that that means that you're playing an incredible, incredibly mechanical game. So essentially what you're doing is you're just kind of waiting for certain kinds of hands. You're waiting until your hand is pretty good, and then I'm just getting you to bet way more money than you should and you know and that that's the way I'm winning. It's it's mechanical. And what I discovered was people don't like winning in

a mechanical way. It's it's almost like they feel like they're No. I don't think it's that they feel like they're cheating. There's not. It's not them making the decision. They're just following the rules, right, So what I would try to do. It's like, yes, it's not that they're not the ones out thinking anyone, Like they're not playing spectacularly.

So what I would try to do is get them to a more meta place, which I think brings us to this what you said about trend following to this more meta place of if you view your job in the game to make the best decisions that you can against the way that the market is behaving, right, because the poker table is just a market, right, so the market is behaving in a certain way. Let's him at a table full of a lot of beginners, it's going to behave differently than if I'm at a table full

of experts. That my job is to come up with the best strategy given the market that I'm playing in. And if I understand that, then it's a form of outsmarting my opponents. To move to this more mechanical version of the game. In the same way that when I played against Derek Sidell, I was willing to say, I'm not going to try to outthink them. I'm just gonna go with luck. And I'm pretty proud of that as

a strategy because it's a meta strategy. It's like, what did you say to you after you you beat him with that strategy? What was the subsequent conversation. I think it was really proud of me. Really, that's that's a great mentor. That's a great mentor who says, so, now the student has become the teacher. And well sort of because I was admitting that I still couldn't think it, yes,

but he was. He was very proud of the I think of the fact that that was the strategy, that I looked at the lay of the land, and you chose the smartest route to give yourself the best possibility of winning. It's still gonna could have come up against you,

but you think playing the other way, you would. And by the way, I get it when when I play against beginners and they win with some hand that like no reasonable person on earth whatever, I have that feeling of like life is unfair and you know, sure there's that blind spot bias rearing it. Yeah. I mean, one of the things I really try to get across in the book is that I know all of this stuff

and I still do it. It's just that I do it a little less and a little less, as you know, just compounds over time into such a higher probability of good results. The other thing, it's because I know him accountable to the people that I have around me, who are going to hold me accountable to bias thinking, who are going to say no, no, no, that's not okay. That I'm much more likely to catch an error, and

I'm much more likely to get it faster. Right. So instead of looking back on twenty year old Annie and say, wow, that was a really crazy decision that she made, or that was a crazy belief that she held, I think that I can get to that conclusion a lot more quickly because I have other people helping me get there. I like that you go back to twenty year old Annie and not thirty year old name. No, listen seriously, Like, how about just like forty five year old Annie, Like

yesterday Annie? Like yesterday Annie was doing really stupid stuff. You know. It's it's amazing that if you stop and look back at your life honestly every five years. Uh. The one thing I could say about being in my fifties. When I went through that exercise in my thirties and forties, it was just mortifying. And now it's merely like that was a bad decision that was a bad process, but it's not like, oh, my god, would idiot you are,

which was what the experience was in the twenties and thirties. Well, I think maybe part of that is because I think that if you managed to survive a long time, whatever it is, in poker or anything else, by the time you're fifty, you have I think you probably have developed some more humility around the rightness and wrongness of your

decisions in the first place. And that's one of the things that I try to point out in the book is that when you view your beliefs is under under construction, you know, in progress, then you don't end up with these full on reversals too. I'm an idiot, right, because you kind of didn't view yourself as a genius in

the first place. And so I think it's a place where you end up, first of all, being much more compassionate to yourself because you sort of realize it's like, okay, but I was only like on it anyway, so now all right, I'm okay. Right. It's like an adjustment to what your level of confidence in the in the whatever the belief is that you hold. And I think that that's just kind of a nicer way to be to yourself.

And then I think that what falls from that, once you sort of view your own beliefs is under construction, is that you are much more compassionate towards other people for their own And still I'm still waiting for that to at least when I'm driving, I'm waiting, well, my wife and so I don't do New Year's resolutions for the exact reason you're implying. The decision isn't. It's all about the process. But there were two processes I wanted

to try and spend a little more effort with. One was just being nicer on Twitter, because there are some people on Twitter who were that started this year around the holidays. People are some people are just jerks on Twitter, and it's like, you know what, that's a bad look. Let me see if I could be less of a jerk. And then the other thing was in real life, Hey

can I be a little nicer on the highway? Just pick up whatever karma points letting people in or But I still find myself constantly going to say, uh, your left directional has been on for about three miles, Either make a left or shut that off. And my wife is like I thought that was going to stop, and it's so hard to make those little change it is. But do you think you're doing it slightly less? Oh?

I am see, And that's what matters. See. That's the thing is that I think that in general, it kind of goes along with that idea of there's only right and wrong, so that if you present me with information that disagrees with me, I can only the only choice I have is right to wrong right. I have to do a full on reversal. If I'm like am six and you present me with information now, I'm like, okay, I'm for It's just an easier way to be right.

So I think that to say I'm gonna I have this idea that I want to be nicer on the role road and to hold yourself to a standard of perfection, you're just setting yourself up to feel bad and and to lack compassion for yourself and for self recrimination. If you say you know what, I'm doing a little better, like that's great. I don't want to make it too easy on myself. The other the other two notes, you're more likely to succeed if you do, though, because it's

the baby with the bathwater. It's like the what the hell effect which Danner ariel It talks about. So it's the difference between Okay, I've decided I'm not going to eat any bread, and then one morning, like there's donuts in the break room and the whole day one calorie and it's and that's because you're you're viewing yourself from the point of perfection. So once you make a little mistake, it tends to it tends to sort of break apart.

Whereas did you say this is this is a work in progress, and all right, I ate half a donut this morning, that's okay, and you just have salad for lunch because you're not. You haven't broken from the perfection that you expected you of yourself. All you were doing was trying to do it less, and it's actually a better way to get there. I think. I love that Arlie's research facility at Duke is called the Institute for Advanced Hindsight or something like that. It's it's just it's

just a hilarious issue. You also mentioned that I'm fascinated by this. If you want to learn something, teach it and over the past, I don't know it's called it a decade, I've been giving presentations around the country, around the world, and each presentation almost always leads to the same sets of questions, and it makes me realize I'm doing a bad job teaching this if I get the

same questions over and over again. And so each presentation leads to the next presentation and a different set of questions, and I'm kind of coming to the point where I'm not sure if it's that I'm a bit teacher or that I'm just provoking people's thoughts that they're going to ask. But when you consistently get the same questions, it makes you think there's something about this presentation, which often has elements of cognitive issues and confirmation bias in it, that

leads people um to the same sort of spot. It's

it's kind of fascinating. So yeah, I think that that is and I think that the analysis is really interesting because it could be that, uh, there are some concepts that are hard to get to within the presentation, and you're kind of anticipating that those questions will come and planning to handle them U in Q and A. I do that some because I don't and and sometimes it's it's that you I mean, this sort of comes to Okay, I could anticipate this question, so let me see if

I can fit that in with the key concepts. But if there are these big concepts that I'm trying to get across and trying to go down the tangent that where I know this question is going to be coming my way, is actually going to distract from the larger concept. Maybe it's you're getting those questions because you've given those them the large concepts in such a great way. I'm not saying that that's true. I'm saying, like, we want

to sort of think about it from both sides. What I've been finding lately is I've been opening my talks with a video of the Pete Carroll play um, and so my questions tend to be people telling me that I'm an idiot and that Pete Carroll actually made a really horrible decision. And even after You've spent half hour explaining why statistically this was not a terrible decision, and you're just focusing on you. What's interesting is that what I didn't say is that Bill Belichick thinks it was

a good call. He no, he said that the formation that that they had had the same defensive formation earlier in the game and it was actually a run protection and so he suspected that Pete Carroll had seen that, and so called the path. Uh. Mike Lombardi, who is a great analyst, actually has worked with Belichick and Bill Walsh and all these guys. Uh, he has it's coming out in September, and it opens with the same play and it discusses it from a formation standpoint as well,

where he sort of comes to the same conclusion. And I kind of mentioned that, so I sort of get myself all there. It's usually people who are Seahawks fans, and so the thing is that, I mean, this is why I said, you know, when you asked me, well, why did you choose trend falling, I said, well, I tried to think of something an investment that has a

lot of emotion attached to it. Is that when people are really emotionally invested in the beliefs that they have, and particularly in the outcome in this case, right, because the Seahawks lots there's no um it's very very hard when even when you're presented with information that might moderate your belief it's just hard to do it. We're just too emotionally lit up to it. It's that system. One

doesn't let it get to system too. Yeah, so there are certain I think that there are many things where someone could get up and give a talk and say all the things that I said, where maybe I would go, you're dumb, that Pete Carroll was was an idiot. It's it's just a matter of like trying to do it less. But I do find that I seem to be getting a lot of that people, people sort of writing into my website and being like, no, that was a horrible call.

Are you seeing these all coming from the Pacific Northwest or what a coincidence? All right, So I have a ton of my favorite questions to ask you, and I know I don't have you forever. Let me jump into some of these things because I think, um, your answers

to this is gonna be fascinating. Tell us the most important thing that people don't know about your background, the most okay, so well about my background or about me now I'll tell you about me now, is that I think that I spent so much of my life in this really competitive environment. Um So when I was younger, uh, when we were playing cards, my brother was two years older than me, which when year seven is like they

might as well. Yeah. Um. And my dad, of course, was a middle aged man, and both of them are very competitive and never let me win. So I would literally like almost every night that we were playing cards, the cards would end up getting thrown against the wall by me at some point. And that was actually my biggest challenge when I was playing poker, was trying to get that emotional component under control, trying to get that competitiveness to a place where I could be smoother and

sort of like accept the outcome. Um, because I would just get I mean, when I was young, I would just become completely unhinged by it. You don't want to be rattled at the table, right. I've gone so completely the other way that since I retired, if I were to go out and play tennis with you, I would

ask you if we could please not keep score? Yeah, because I just want to play and like have both of us get better the people that I play with I always bring to my tennis lessons because I just want to, like, I'm so much more focused on this kind of win win, which is one of the reasons why I think I love the process of writing the book and giving talks because it feels it's like so not competitive, you know, and I feel like I've sort

of like shed a lot of that now. I mean, I don't know if people around me would agree, but I'm certainly it's certainly left. That's very interesting. You mentioned some of your early mentors. Give us a little background on them. Yeah, so I got plugged into through my brother, this amazing group of players in New York. Uh so they all they all started off playing in New York. They had all come from this game's background. Um. Some like Eric Sdel was an amazing uh backgammon player and

he was also an options trader. And then you know, my brother came from really gotten there through chas. You had Dan Harrington, who was won the World Series of Poker UM main event, I mean, world champion, amazing player, has written some of the most incredible books on poker that you can get Harrington on hold them if anybody's interested in learning, go get those books. UM guy named Jason Lester who was one of the best back men players,

and I got plugged into them as mentors. Now what's amazing is how much money collectively that group of players has made. I have it in a in a end note in the book, but just so that you know, Eric Sidell alone has earned thirty eight million dollars in tournament poker. So these were my mentors and and Eric. I think uh is particularly prominent in the book because when I talked about how do you have people watch your back? He's the one who said this thing to me.

I had known him since I was sixteen, when I wasn't a poker player, and we were just like friends because he was friends with my brother. Then all of a sudden, I'm a poker player and I walk up to him like he's still my buddy. One day and I just start moaning to him about this horrible luck that I had had and I can't believe this. I got so unlucky. It's such a bad beat. Blah blah blah blah blah. And he laid it out for me. Boom, He said, why are you even telling me this story?

Like do you think I need like your emotional hard luck story? Like voiced it upon me. I've lost with good hands too, Like do you have a question, Because if you have a question, I'll talk to you all day. But I just don't want to hear about your bad luck stories, like literally there's nothing to be learned from it. And and while it sounds a little bit harsh, that was the most important moment of mentorship that I've ever received in my life. That's quite quite interesting. So anyone

else influence how you approached the art of gambling? Yeah, I mean I just obviously that group, And then within poker there was also a guy named John Hannigan who's actually mentioned in the book, who was someone I watched really carefully. Um some I think that some of the things most important lessons I learned from people who didn't who probably didn't understand that they were mentoring me in

some way where I was just learning from them. And in particularly important lessons for me were people who played in ways that my group had not taught me to play. So they were playing strategies that were very different from the kinds of strategies that I I was being sort

of taught and that we're being fostered than me. And the initial feeling on my part of they must be really bad, like clearly they're just idiots, and they're winning because they're just getting lucky, because just simply because I didn't understand their strategies, and that shift of learning to sort of dig into their strategies and figure out why it was working for them and what they were doing, even though it wasn't something that I was coming and

thinking was sort of the right play, UM was really actually important. It's that openness to things that you don't understand that just because you don't understand them or they're not within your wheelhouse doesn't mean they're bad. Sometimes it does, but it doesn't always. And that was I think some of the most important membership mentorship that I received, And I don't think those people know that they mentored me. Tell us about a time you failed and what you

learned from the process. Oh, my gosh, which time? I mean so many? I mean, you know, I feel like for a long time I sort of carried around not finishing my PhD as a failure. For sure. UM, I think that that was a lesson in compassion towards myself. UM, you fail all the time at the poker table, I think you have to learn how to process that. I had a startup UM that I was partnering called Epic Poker that we started right at the end of UM.

The poker boom right. So this was right around I think we started the company in two thousand ten and we were trying to create sort of the PGA of poker, and that company failed and it was it was awful. Like I obviously I really wanted it to succeed, and whenever a company fails, there's vendors that are angry. Some of the players were angry. UM, and that was certainly

a lot to process. What I try to take from every single time that I fail, whether it's the company or failing on a particular poker hand, or feeling like I failed in a tournament because maybe there were decisions that I could made would have been better, I made a big mistake that caused me to lose, or whatever it might be. Is to say, the only real failure is to not to take lessons from it, to figure out to start trying to parse out what was luck,

what was skill? What could I change in the future, How do I find compassion for myself? How do I find compassion for other people? UM? And that's what I'm always trying to work on, is that. UM And hopefully I'm I'm doing an okay job at that. That's quite quite interesting. What do you do to stay mentally or physically fit outside of work. Well, I think that the you know, for me, I think that mental and physical

fitness actually go hand in hand. Um. I think that the average view of a poker player is someone who's you know, overweight, with advisor and a cigar or something like that. I'm not sure, but actually most of the really uh top poker players are not that at all and are actually in pretty good physical condition. So I play a lot of tennis, I do a lot of yoga. I do something called solid core, which is like uh, sort of pilates on steroids UM, and I spin UM.

So I am working out actually more than seven times a week because I'm often at layering yoga UM on top of that. And I think that that's just really important for mental fitness. You're sitting down at a poker table or doing work like what you do in investing, it's like you're doing math problems all day. You go open up a textbook and do math problems all day. It's really tiring. It could be physically exhaust it's physically exhausting.

So I think if your physical self isn't in good shape, then your mental self you know you can't be sharp. If a millennial or recent college grad came to you and said they're thinking about a career as a professional poker player, what sort of advice would you give them. I think that now the advice I would give, because it's been on television, so I think it's considered a little more glam is to know what you're getting into.

I mean, it's a grind. You have to put in your hours because the amount of money you're gonna make is gonna be tired to how many hands you get in. So it's long hours. It's tiring. Um. It's it's physically exhausting to be in a casino. It's obviously piped in stale air. There aren't windows, so you're not getting any sunlight. Um, and it's mentally really grueling. So that's number one. Uh, And then number two to recognize that it's a very small portion of people that can really make their living

at it. I think that probably the advice I would give would be the same as someone who came and said, I'm going to go make my living being a day trader. It's like, great, there's some people who are amazing at that, and they're amazing at high frequency trading, and they do an incredible job of it. Just to understand that most people don't right and they go broke. And so just go in there with open eyes as to what you're getting,

as to what your chances of success are. It's it's not that I don't think that you personally can succeed, just know, um, and be prepared for something that's physically and mentally really grueling. And my final question, what is it that you know about poker, playing, gambling, uh, statistic and luck today that you wish you knew twenty years ago. I think essentially everything that I wrote in the book. When I first started playing, I think that I was I think that I thought that I was much better

than I actually was. Um, I thought I was hot, you know yeah, uh, you know I describe this scene and which is I remember so well? You know, so remember I said, my brother wrote all these hands down on a napkin, and my brother was a world champion player, and I somehow thought like I was like way better than these people in the game that I was playing with because I had my napkin from my brother. That's like as Dunning Krueger, as you could get right and people.

When I saw people play hands that weren't on my brother's list, I was just like, Wow, that person must be an idiot. They're playing a hand that isn't on my brother's list, and I just said, I'm so embarrassed by that. Oh my gosh. So when I first started playing,

I wish I had known how little I knew. I wish I had known that I was not even close, like I was a seventeen universes away from being any kind of expert, and that here's a really importan thing that I wish that I had known that the worst player at the table has something to teach you, and that thing that they're going to teach you is really really important. And it took me a long time to figure that one out. That's quite fascinating. We have been

speaking with Annie Duke. She is the author of Thinking in Bets, Making Smarter decisions when you don't have all the facts. If you enjoy this conversation, be sure and check Up an Inch or Down an Inch on Apple iTunes and you could see any of the other At this point almost two hundred podcasts that we've had previously. We enjoy your comments. And feedback and suggestions right to

us at m IB podcast at Bloomberg dot Net. I would be remiss if I did not thank our staff who helps to put together UH this podcast each week. Medina Parwana is our producer and audio engineer. Taylor Riggs is our booker. Mike bat Nick is our head of research. I'm Barry Retolts. You've been listening to Masters in Business on Bloomberg Radio. Tempting, tempted to so