In this special episode, the roles are reversed. as I step into the guest seat to explore the intriguing world of Bayesian stats. Originally aired as episode 793 on the fantastic Super Data Science podcast hosted by John Crone, this conversation is too good not to share with all of you here on learning Bayesian statistics. So join us as we delve into how Bayesian methods elegantly handle complex problems, make efficient use of prior knowledge and excel with limited data.
the foundational concepts of patient statistics, highlighting their distinct advantages over traditional methods, particularly in scenarios fraught. with uncertainty and sparse data. A highlight of our discussion is the application of Gaussian processes where I explain their versatility in modeling complex, non -linear relationships in data. I share a fascinating case study involving an NGO in Estonia illustrating how Bayesian approaches can transform limited polling data into profound insights.
So whether you're a seasoned statistician or just starting out, this episode is packed with practical advice on embracing Bayesian stats and of course I strongly recommend you follow the Super Data Science Podcast. It's really a can't -miss resource for anyone passionate about the power of data. This is Learning Vision Statistics, episode 113, originally aired on the Super Data Science Podcast.
Welcome to Learning Bayesian Statistics, a podcast about Bayesian inference, the methods, the projects, and the people who make it possible. I'm your host, Alex Andorra. You can follow me on Twitter at alex -underscore -andorra. like the country. For any info about the show, learnbasedats .com is Laplace to be. Show notes, becoming a corporate sponsor, unlocking Bayesian Merge, supporting the show on Patreon, everything is in there. That's learnbasedats .com.
If you're interested in one -on -one mentorship, online courses, or statistical consulting, feel free to reach out and book a call at topmate .io slash alex underscore and dora. See you around, folks. and best patient wishes to you all. And if today's discussion sparked ideas for your business, well, our team at PIMC Labs can help bring them to life. Check us out at pimc -labs .com. Hello my dear Vagans! A quick note before today's episode, STANCON 2024 is approaching!
It's in Oxford, UK this year from September 9 to 13 and it's shaping up to be an incredible event for anybody interested in statistical modeling and vaginal inference. Actually, we're currently looking for sponsors to help us offer more scholarships and make STANCON more accessible to everyone and we also encourage you to buy your tickets as soon as possible. Not only will this help with making a better conference, but this will also support our scholarship fund.
For more details on tickets, sponsorships, or community involvement, you'll find the Stencon website in the show notes. We're counting on you. Okay, on to the show now. Alex, welcome to the Super Data Science podcast. I'm delighted to have you here. Such an experienced podcaster. It's going to be probably fun for you to get to be the guest on the show today. Yeah. Thank you, John. First, thanks a lot for having me on. I knew about your podcast.
I was both honored and delighted when I got your email to come on the show. I know you have had very... Honorable guests before like Thomas Vicky. so I will try to, to, to be on board, but, I know that it's going to be hard. Yeah. Thomas, your co -founder at, Pi MC labs is, was indeed a guest. was on episode number 585. but that is not what brought you here. Interestingly, the connection. So you asked me before we started recording how I knew about you.
And so a listener actually suggested to you as a guest. So. Doug McLean. Thank you for the suggestion. Doug is lead data scientist at Tesco bank in the UK. And he reached out to me and said, can I make a suggestion for a guest? Alex Andora, like the country, I guess you say that you say that. Cause he put it in quotes. He's like, Andora, like the country hosts the learning patient statistics podcast. It's my other all time favorite podcast. So there you go. my God. Doug, I'm blushing.
says he'd be a fab guest for your show and not least because he moans from time to time about not getting invited onto other podcasts. Did I? my God. I don't remember. But maybe that was part of a secret plan, Maybe a secret marketing LBS plan and well. That works perfectly. When I read that, I immediately reached out to you to see if you'd want to go, but that was so funny.
And he does say, says, seriously though, he'd make a fab guest for his wealth of knowledge on data science and on Bayesian statistics. And so, yes, we will be digging deep into Bayesian statistics with you today. you're the co -founder and principal data scientist of the popular Bayesian statistical modeling platform, PI MC, as we already talked about with your co -founder Thomas Wiki. That is an excellent episode. If you want to go back to that and get.
different perspective, obviously different questions we've made sure. But so if you're really interested in Bayesian statistics, that is a great one to go back to. Yeah, in addition to that, you obviously also have the Learning Bayesian Stats podcast, which we just talked about, and you're an instructor on the educational site, Intuitive Bayes. So tons of Bayesian experience. Alex, through this work, tell us about what Bayesian methods are and what makes them so powerful and versatile?
Yeah. so first, thanks a lot. Thanks a lot, dog, for the recommendation and for listening to the show. am, I am absolutely honored. and, yeah, go and listen again to Thomas's episode. Thomas is always a great guest. So I definitely recommend anybody to, to go and listen to him. now what about Bayes? Yeah. You know, it's been a long time since someone has asked me that, because I have a Bayesian podcast. Usually it's quite clear I'm doing that. people are like afraid to ask it at some point.
So instead of giving you kind of like a, because our two avenues here, usually I could give you the philosophical answer and why epistemologically Bayes stats makes more sense. but I'm not going to do that. That sounds so interesting. Yeah, it is, we can go into that. I think a better introduction is just a practical one.
And that's the one that most people get to know at some point, which is like you're working on something and you're interested in uncertainty estimation and not only in the point estimates and your data are crap and you don't have a lot of them and they are not reliable. What do you do? And that happens to a lot of PhD students. That happened to me when I started trying to do electoral forecasting.
was at the time working at the French Central Bank doing something completely different from what I'm doing today. But I was writing a book about the US at the time, 2016 it was, and it was a pretty consequential election for the US. was following it. really, really closely. And I remember it was July 2016 when I discovered 538 models. And then the nerd in me was awoken. It was like, my God, this is what I need to do.
You know, that's my way of putting more science into political science, which was my background at the time. And when you do electoral forecasting polls are extremely noisy. They are not a good representation of what people think, but they are the best ones we have. are not a lot of them, at least in France, in the US much more. It's limited. It's not a reliable source of data basically. And you also have a lot of domain knowledge, which in the Bayesian Royal realm, we call prior information.
And so that's a perfect setup for Bayesian stats. So that's basically, I would say what Bayesian stats is. And that's the powerful, the power of it. You don't have to rely only on the data because sure you can let the data speak for themselves, but what if the data are unreliable? Then you need something to guard against that and patient stats are a great way of doing that. And the cool thing is that it's a method. It's like you can apply that to any topic you want, any field you want.
that's what... I've done at PMC Labs for a few years now with all the brilliant guys who are over there. You can do that for marketing, for electoral forecasting, of course. Agriculture, that was quite ironic when we got some agricultural clients because historically, agriculture is like the field of frequency statistics. That's how Ronald Fisher developed the p -value, the famous one. So when we had that, we're like, yes, we got our revenge.
And of course, it's also used a lot in sports, sports modeling, things like that. So yeah, it's like that's the practical introduction. Nice. Yeah. A little bit of interesting history there is that sub -Asian statistics is an older approach than the frequentist statistics that is so common and is the standard that is taught in college, so much so that is just called statistics.
You do an entire undergrad in statistics and not even hear the word Bayesian because Fisher so decidedly created this monopoly of this one kind of approach, which for me, learning for Quenta statistics say, I think I guess it was first year undergrad in science that I studied and in that first year course, that idea of a P value always seemed odd to me. Like how is it that there's this art?
This is such an arbitrary threshold of significance to have it be, you know, that this is a one in 20 chance or less that this would be observed by chance alone. And this means that therefore we should rely on it, especially as we are in this era of large data sets and larger and larger and larger data sets.
You can have no meaningful if with very large data sets like we typically deal with today, no matter, you're always going to get a significant P value because the slightest tiny change, if you take, you know, web scale data, everything's going to have be statistically significant. Nothing won't be. so it's such a weird paradigm. And so that was discovering Bayesian statistics and machine learning as well. And seeing how Those areas didn't have P values interested me in both of those things.
it's a, yeah, Fisher. It's interesting. mean, I guess with small data sets, eight, 16, that kind of scale, guess it kind of made some sense. And you know, you pointed out there, I think it's this prior that makes Bayesian statistics so powerful being able to incorporate prior knowledge, but simultaneously that's also what makes for Quentus uncomfortable. They they're like, we want only the data.
As though, you know, the particular data that you collect and the experimental design, there are so many ways that you as the human are influencing, you know, there's no purity of data anyway. And so priors are a really elegant way to be able to adjust the model in order to point it in the right direction.
And so a really good example that I like to come to with Bayesian statistics is that you can You can allow some of your variables in the model to tend towards wider variance or narrower variance. So if there are some attributes of your model where you're very confident, where you know this is like, you know, this is like a physical fact of the universe. Let's just have a really narrow variance on this and the model won't be able to diverge much there.
But that then gives a strong focal point within the model. around which the other data can make more sense, the other features can make more sense, and you can allow those other features to have wider variance. And so, I don't know, this is just one example that I try to give people when they're not sure about being able to incorporate prior knowledge into a model. Yeah, yeah, no, these are fantastic points, John. So, yeah, to build a net, I'm... I'm a bit, of course, I'm a nerd.
So I love the history of science. I love the epistemological side. A very good book on that is Bernoulli's Fallacy by Aubrey Clayton. Definitely recommend his book. He was on my podcast, episode 51. So if people want to give that a listen. Did you just pull that 51 out from memory? Yeah, yeah, I kind of know like, but I have less episodes than you. So it's like, you know, each episode is like kind of my baby. So I'm like, yeah, 51 is Aubrey Clayton. Yeah. my goodness. That's crazy.
That's also how my brain works. numbers. But yeah. And actually episode 50 was with Sir David Spiegel Halter. I think the only night we got on the podcast and David Spiegel Halter exceptional. exceptional guest, very, very good pedagogically. Definitely recommend listening to that episode two, which is very epistemologically heavy so far for people who like that, the history of science, how we got there. Because as you were saying, is actually older than Stantz, but people discovered later.
So it's not because it's older, that's better, right? But it is way older actually by a few centuries. So yeah, fun stories here. could talk about that still, but to get back to what you were saying, also as you were very eloquently saying, data can definitely be biased. Because that idea of like, no, we only want the data to speak for themselves. as I was saying, yeah, what if the data are unreliable? But as you were saying, what if the data are biased? And that happens all the time. And worse.
I would say these biases are most of the time implicit in the sense that either they are hidden or most of the time they just like you don't even know you are biased in some direction most of the time because it's a result of your education and your environment. So the good thing of priors is that it forces your assumptions, your hidden assumptions to be explicit.
And that I think is very interesting also, especially when you work on models which are supposed to have a causal explanation and which are not physical models, but more social models or political scientific models. Well, then it's really interesting to see how two people can have different conclusions based on the same data. It's because they have different priors. And if you force them to explicit these priors in their models, they would definitely have different priors.
then... then you can have a more interesting discussion actually, think. So there's that. And then I think the last point that's interesting also in that, like why would you be interested in this framework is that also, causes are not in the data. Causes are outside of the data. The causal relation between X and Y, you're not going to see it in the data because if you do a regression of education on income, you're going to see an effect of education on income.
But you as a human, you know that if you're looking at one person, the effect has to be education has an impact on income. But the computer could like might as well just do the other regression and regress income and education and tell you, income causes education. But no, it's not going that way. the statistical relationship goes both ways, but the causal one only goes one direction. And that's a hidden reference to my favorite music band.
But yeah, it only goes one direction, and it's not in the data. And you have to have a model for that. And a model is just a simplification of reality. We try to get the simple enough model that's usually not simple, but that's a simplification. If you say it's a construction and simplification, that's already a prior in a way. you you might as well just go all the way and explicit all your priors. Well said. Very interesting discussion there.
You used a term a number of times already in today's podcast, which maybe is not known to all of our listeners. is epistemology? What does that mean? right. Yeah, very good question. Yeah. So epistemology in like, in a sense that's the science of science. It's understanding how we know what we say we know. So, for instance, how do we know the earth is round? How do we know about relativity? Things like that. it's the scientific discipline that's actually very close to also philosophy.
That's, think, actually a branch of philosophy. And that's trying to... come up with methods to understand how we can come up with new scientific knowledge. And by scientific here, we usually mean reliable and reproducible, but also falsifiable. Because for hypothesis to be scientific, it has to be falsifiable. so yeah, basically that's that. Lots of extremely interesting things here, but yeah, that's like basically how do we know?
what we know, that's the whole trying to define the scientific method and things like that. Going off on a little bit of a tangent here, but it's interesting to me how I think among non -scientists lay people in the public. Science is often seen to be infallible, as though science is real. Science is the truth. There's a lot of, since that 2016 election, there are lots of, people have lawn signs in the US that say, that basically have a list of liberal values, most of which I'm a huge fan of.
And of course I like the sentiment, this idea, you know, that they're supporting science on this sign, on the sign as well. But it says the sign, the way that they phrase it is science is real. And the implication there for me, every time I see the sign is that, you know, and I think that could be, for example, related to vaccines, I think, you know, around, you know, there was a lot of conflict around vaccines and what their real purpose is and, you know, and then so.
the lay liberal person is like, you know, this is science, know, trust science, it's real. Whereas from the inside, it's, you you pointed it out already there, but it's this interesting irony that the whole point of science is that we're saying, I'm, I'm, I'm never confident of anything. I'm always open to this being wrong. Yeah. Yeah. No, exactly. and I think that's, that's kind of the distinction.
That's often made in epistemology actually between science on one hand and research on the other end, where research is science in the making. Science is like the collective knowledge that we've accumulated since basically the beginning of modern science, at least in the Western hemisphere, so more or less during the Renaissance. Then research is people making that science because... people have to do that and how do we come up with that?
so, yeah, like definitely I'm one who always emphasizes the fact that, yeah, now we know the Earth is round. We know how to fly planes, but there was a moment we didn't. And so how do we come up with that? And actually, maybe one day we'll discover that we were doing it kind of the wrong way, you know, flying planes, but it's just like, for now it works. We have the best model that we can have right now with our knowledge. But maybe one day we'll discover that there is a way better way to fly.
And it was just there staring at us and it took years for us to understand how to do that. yeah, like as you were saying, but that's really hard line to walk because you have to say, yeah. Like these knowledge, these facts are really trustworthy, but you can never trust something 100 % because otherwise mathematically, if you go back to base formula, you actually cannot update your knowledge.
you, if you have a 0 % prior or 1 % prior, like mathematically, you cannot apply base formula, which tells you, well, based on new data that you just observed the most rational way of updating your belief is to believe that with that certainty. But if you have zero or 100%, it's never going to be updated. So you can say 99 .9999 % that what we're doing right now by flying is really good. But maybe, like, you never know. There is something that will appear. And physics is a real...
We've all seen UFOs, Alex. We know that there's better ways to fly. Yeah. Exactly. Yeah, but yeah, think physics is actually a really good field for that because it's always evolving and it's always coming up with really completely crazy paradigm shifting explanation like relativity, special relativity, then general relativity just a century ago that didn't exist. And now we start to understand a bit better, but even now we don't really understand how to how to blend relativity and gravity.
so that's extremely interesting to me. But yeah, I understand that politically from a marketing standpoint, it's hard to sell, but I think it's shooting yourself in the foot if you're saying, yeah, is always like, science works, I agree, science works, but it doesn't have to be 100 % true and sure. for it to work. That's why placebo works. Placebos work, right? It's just something that works even though it doesn't have any actual concrete evidence that it's adding something, but it works.
yeah, like I think it's really shooting yourself in the foot by saying that no, that's 100%. Like if you question science, then you're anti -science. No. Actually, it's the whole scientific methods to be able to ask questions all the time. The question is how do you do that? Do you apply the scientific method to your questions or do you just question anything like that without any method? And just because you fancy questioning that because it goes against your belief to begin with.
So yeah, that's one thing. And then I think another thing that you said I think is very interesting is, unfortunately, I think the way of teaching science and communicating around it, is not very incarnated. It's quite dry. You just learn equations and you just learn that stuff. Whereas science was made by people and is made by people who have their biases, who have extremely violent conflicts. Like you were saying, Fisher was just a huge jerk to everybody around him.
I think it would be interesting to get back to a bit of that human side to make science less dry and also less intimidating thanks to that. Because most of the time when I tell people what I do for a living, they get super intimidated and they're like, my God, yeah, I hate math, I hate stats and stuff. But it's just numbers. It's just a language. it's a bit dry. For instance, if there is someone who is into movies, who does movies in your audience.
I want to know why there is no movie about Albert Einstein. There has to be a movie about Albert Einstein. Like not only huge genius, but like extremely interesting life. Like honestly, it makes for great movie. was working in a a dramatized biopic. mean? Yeah. Yeah. I mean, it's like his life is super interesting. Like he revolutionized the field of two fields of physics and actually chemistry.
In 1905, it's like his big year, and he came up with the ideas for relativity while working at the patent bureau in Bern in Switzerland, which was an extremely boring job. In his words, it was an extremely boring job. Basically, having that boring job allowed him to do that being completely outside of the academic circles and so on. It's like he makes for a perfect movie. I don't understand why it's not there. And then I sing on the cake. He had a lot of women in his life.
So it's like, you know, it's perfect. Like you have you have the sex you have, you have the drama, you have revolutionizing the field, you have Nobel prizes. And he and then he became a like a pop icon. I don't know where the movies. Yeah, it is wild. Actually, now that you pointed out, it's kind of surprising that there aren't movies about him all the time. Like Spider -Man. Yeah, I agree. Well, there was one about Oppenheimer last year. Maybe that started to trend. see.
Yeah. So in addition to the podcast, you also, I mentioned this at the outset, I said that your co -founder and principal data scientist of the popular Bayesian stats modeling platform, PyMC. So like many things in data science, it's uppercase P, lowercase y for Python. What's the MC, PyMC, one word, and C are capitalized. Yeah. So it's very confusing because it stands for Python and then MC is Monte Carlo. So I understand. But why Monte Carlo?
It's because it comes from Markov chain Monte Carlo. So actually it should be pie MCMC or pie MC squared, which is what I'm saying since the beginning. anyways, yeah, it's actually, it's actually buying C squared. so for Markov chain Monte Carlo and Markov chain Monte Carlo is one of the main ways that all of their algorithms now, new ones, but like the blockbuster algorithm to run a patient models is to use MCMC.
Yeah. So in the same way that stochastic gradient descent is like the de facto standard for finding your model weights in machine learning, Markov chain Monte Carlo is kind of the standard way of doing it with a Bayesian network. Yeah. Yeah. Yeah. And, so now there are newer versions, more efficient versions. That's, that's basically the name of the game, right? Making the efficient, the algorithm more and more efficient. but the first algorithm.
days back, I think it was actually invented during the project Manhattan. during the world during World War Two. Game of the day. Yeah. And lots of physicists actually, statistical physics is a film that's contributed a lot to MCMC. so yeah, like physicists who came to the field of statistics and trying to make the algorithms more efficient for their models. And yeah, so they buy They have contributed a lot.
The field of physics has contributed a lot of big names and people to great leaps into the realm of more efficient algorithms. I don't know who your audience is, but that may sound boring. Yeah, the algorithm, it's like the workhorse. But it's extremely powerful.
And that's also one of the main reasons why patients' statistics are increasing in popularity lately because I'm going to argue that it's always been the best framework to do statistics, to do science, but it was hard to do with pen and paper because the problem is that you have a huge nasty integral on the numerator, on the denominator, sorry. And this integral is not computable by pen and paper. So for a long, long time, patient statistics combined to features, you know, like campaigns.
PR campaigns, patients S6 was relegated to the margins because it was just super hard to do. so for other problems, other than very trivial ones, it was not very applicable. But now with the advent of personal computing, you have these incredible algorithms like, so now most of time it's HMC, Hamiltonian Monte Carlo. That's what we use under the hood with PIMC. But if you use Stan, if you use NumPyro, it's the same.
And thanks to these algorithms, now we can make extremely powerful models because we can approximate the posterior distributions thanks to, well, computing power. A computer is very good at computing. I think that's why it's called that. Yes. And so that reminds me of deep learning.
It's a similar kind of thing where the applications we have today, like your chat GPT or whatever your favorite large language model is these amazing video generation like Sora, all of this is happening thanks to deep learning, which is an approach we've had since the fifties, certainly not as old as Bayesian statistics, but similarly it has been able to take off with much larger data sets and much more compute. Yeah. Yeah. Yeah. Yeah, very good point.
And I think that's even more the point in deep learning. for sure. Because Beijing stats doesn't need the scale, but the way we're doing deep learning for now definitely need the scale. Yeah, yeah. Scale of data. Yeah, exactly. Yeah, sorry. Yeah, the scale. Because there two scales, data and... Yeah, you're right. Yeah, and for like model parameters.
And so that has actually, I mean, tying back to something you said near the beginning of this episode is that actually one of the advantages of Beijing statistics is that you can do it with very few data. Yeah. maybe fewer data than with a frequentist approach or machine learning approach. Because you can bake in your prior assumptions and those prior assumptions give some kind of structure, some kind of framework for your data to make an impact through. Yeah, completely.
So for our listeners who are listening right now, if they are keen to try out Bayesian statistics for the first time, why should they reach for PyMC? Which, as far as I know, is the most used. Bayesian framework, period. And certainly in Python. and then the second I'm sure is Stan. yeah. Yeah. And, so, yeah, why, should somebody use pyMc and maybe even more generally, how can they get started if they haven't done any Bayesian statistics before at all? Yeah. Yeah. Yeah. Fantastic question.
I think it's a, yeah, it's a very good one because, that can also be very intimidating. And actually that can be a paradox of choice. know, where now we're lucky to live in a world where we actually have a lot of probabilistic programming languages. So you'll see that sometimes that's called PPL and that's what's a PPL. It's basically PINC. It's a software that enables you to write down Bayesian models and sample from them. Okay. So it's just a fancy word to say that.
Yeah, my main advice is don't overthink it. Like if you're proficient in R, then probably you want to try, I would definitely recommend trying BRMS first because it's built on top of Stan and Stan is extremely good. It's built by extremely good modelers and statisticians. Lots of them have been on my podcast. So if you're curious, just, just go there and you go on the website, you look for Stan and you'll get a lot of them.
the best one is most of the time, Andrew Gellman, absolutely amazing to have him on the show. He, he always explains stuff extremely clearly. but I also had Bob Carpenter, for instance, Matt Hoffman. so anyways, if you know, or. yeah. Have you ever had Rob Tran Gucci on the show, or do you know who he is? I know, but I have never had him on the show. So, but I'd be happy to. Yeah. If you know him, I'll make an introduction for you. He was on our show in episode number 507.
And that was our first ever Beijing episode. And it was the most popular episode of that year, 2021, the most popular episode. And it was interesting because also up until that time, at least with me hosting, 2021 was my first year hosting the show. And it was by far our longest episode. I was like, that was kind of concerning for me. was like, this was a super technical episode, super long. I was like, how is this going to resonate? It turns out that's what our audience loves.
And that's something we've been leaning into a bit in 2024 is more technical, longer. Well, that's good to know. Yeah. I'll make an intro for Rob. Anyway, you were saying I could do an intro for you. Yeah, I know. But yeah. Great, great interruption for sure. I'm happy to have that introduction made. a lot. Yeah, so I was saying, if you're proficient in R, definitely give a try to BRMS. It's built on top of Stan. Then when you outgrow BRMS, go to Stan.
If you love Stan, but you're using Python, there is PyStan. I've never used that personally, but I'm pretty sure it's good. and then, but I would say if you're proficient in Python and don't really want to go to R then yeah, like you probably want to give a try to, to PIMC or to NumPyro. You know, give that a try, see what, what resonate most with you, the, API most of the time, because if you're going to make models like that, you're going to spend a lot of time on your code and on your models.
And, as most of your audience probably know, like The models always fail unless it's the last one. So, yeah, you have to love really the framework you're using and find it intuitive. Otherwise, it's going to be hard to keep it going. If you're really, really a beginner, I would also recommend on the Python realm, give a try to Bambi, which is the equivalent of BRMS, but in Python. So Bambi is built on top of Climacy. And what it does, it does a lot of the choices for you.
It makes a lot of the choices for you under the hood. So priors, stuff like that, which can be a bit overwhelming to beginners at the beginning. But then when you outgrow Bambi, you want to make more complicated models, then go to BiMC. Bambi, that's a really cute name for a model that's just like, it just drops out of its mother and can barely stand up straight. Yeah, And the guys working on Bambi, so Tommy Capretto, Osvaldo Martino. So they are like, yeah, really great guys.
Both Argentinians, actually. And yeah, like they are fun guys. I think the website for Bambi is bambinos .github .com. yeah, like these guys. These guys are fun. But yeah, it's definitely a great framework. And actually, this week, we released with Tommy Capretto and Ravin Kumar. We actually released an online course, our second online course that we've been working on for two years. So we are very happy to have released it. But we're also very happy with the course. That's why it took so long.
It's a very big course. And that's exactly what we do. We take you from beginner. We teach you Bambi, teach you Pinsene and you go up until advanced. That's called advanced regression. So we teach you like all things regression. What's the course called? Advanced regression. Yeah. Advanced regression on the intuitive base platform that you were kind enough to mention at the beginning. Nice. Yeah. I'll be sure to include that in the show notes.
And so even though it's called advanced regression, you start us off with an introduction to Bayesian statistics and we start getting our with Bambi before moving on to PyMC, yeah? Yeah, yeah, yeah. So you have a regression refresher at the beginning. If you're a complete, complete beginner, then I would recommend taking our intro course first, which is really here. It's really from the ground up. The advanced regression course, well, ideally you would do that after the intro course.
But if you're already there in your learning curve, then you can start with the intro course. It makes a bit more assumption on on the student's part, like, yeah, they have heard about Bayesian stats. They are aware of the ideas of priors, likelihood, posteriors. But we give you a refresher about the classic progression. it's like when you have a normal likelihood. And then we teach you how to generalize that framework to data that's not normally distributed. And we start with BAMBEE.
We show you how to do the equivalent models in PIMEC. And then at the end, the model became, becomes, become like much more complicated, then we just show it in point C. Nice. That is super, super cool. I hope to be able to find time to dig into that myself soon. It's one of those things. yeah. You and I were lamenting this before the show, podcasting of itself can take up so much time on top of, in both of our cases, we have full -time jobs. This is something that we're doing as a hobby.
And it means that I'm constantly talking to amazingly interesting people like you who have developed fascinating courses that I want to be able to study. And it's like, when am going to do that? Like book recommendations alone. Like I barely get to read books anymore. That was something like since basically the pandemic hit. I, and it's, it's so embarrassing for me because I, I identify in my mind as a book reader. And sometimes I even splurge.
I'm like, wow, I've got to get like these books that I absolutely must read. And they just collect in stacks around my apartment. Like, yeah. Yeah. Yeah. I mean, that's hard for sure. yeah, it's something I've also been trying to, get under control a bit. yeah, like I find, so a guy who does good work, I find on that is, can you port Yes, Cal Newport, of course. I've been collecting his books too. Yeah, that's the irony. So he's got a podcast.
don't know about you, but me, I listen to tons of podcasts. So the audio format is really something I love. So podcasts and audio books. yeah, that can be your entrance here. Maybe you can listen to more books if you don't have time to write. Yeah, it's an interesting... don't really have a commute. and I often, use like, you know, when I'm traveling to the airport or something, I use that as an opportunity to like do catch up calls and that kind of thing. So it's interesting.
I, I, I almost listened to no other podcasts. The only show I listened to is last week in AI. I don't know if you know that show. Yeah. Yeah. Yeah. Great show. I like them a lot. put a lot of work into Jeremy and Andre do a lot of work to get. Kind of all of the last week's news constant in there. so it's impressive. It allowed me to flip from being this person where prior to finding that show and I found it cause Jeremy was a guest on my show. was an amazing guest by the way.
don't know if he'd have much to say about Bayesian statistics, but he's an incredibly brilliant person is so enjoyable to listen to. and, and someone else that I'd love to make an intro for you. He's, he's become a friend over the years. Yeah, for sure.
But yeah, last week in AI, they, I don't know why I'm talking about it so much, but they, I went from being somebody who would kind of have this attitude when somebody would say, if you heard about this release or that, or, I'd say, you know, just because I work in AI, I can't stay on top of every little thing that comes out. And now since I started listening to last week in AI about a year ago, I don't think anybody's caught me off guard with some, with some new release. I'm like, I know.
Yeah, well done. Yeah, that's good. Yeah, but that makes your life hard. Yeah, for sure. If you don't have a commute, come on. But I'd love to be able to completely submerge myself in Bayesian statistics. is a life goal of mine, is to be able to completely, because while I have done some Bayesian stuff and in my PhD, I did some Markov chain Monte Carlo work. And there's just obviously so much flexibility and nuance to this space. can do such beautiful things. I have a huge fan of Bayesian stats.
And so yeah, it's really great to have you on the show talking about it. So, Pi MC, which we've been talking about now, kind of going back to our, back to our thread. Pi MC uses something called Pi tensor to leverage GPU acceleration and complex graph optimizations. Tell us about PyTensor and how this impacts the performance and scalability of Bayesian models. Yeah. Great question. basically the way PyMAC is built is we need a backend.
And historically this has been a complicated topic because the backend, then we had to do the computation. Otherwise you have to do the computations in Python. And that's slower than doing it in C, for instance. And so we have still that C backend. That's kind of a historical remnant, but more and more we're using. when I say we, I don't do a lot of PyTensor code to be honest. mean, contributions to PyTensor. I mainly contribute to PyC. PyTensor is spearheaded a lot by Ricardo Viera. Great.
great guy, extremely good modeler. basically the idea of PyTensor is to kind of outsource the computation basically that PymC is doing. And then, especially when you're doing the sampling, PyTensor is going to delegate that to some other backends. And so now instead of having just the C backend, you can actually sample your PIMC models with the number backend. How do you do that? You use another package that's called nutpy that's been built by Adrian Seybold, extremely brilliant guy again.
I'm like surrounded by guys who are much more brilliant than me. And that's how I learned basically. I just ask them questions. That's what I feel like in my day job at Nebula, my software company. just like, Yeah, sorry. I'm just completely interrupting you. Yeah, no, same. And so, yeah. So Adrian basically re -implemented HMC with NutPy, but using Numba and Rust.
And so that goes way faster than just using Python or even just using C. And then you can also sample your models with two other backends that we have that's enabled by PyTensor that then basically compiles the graph of the model and then delegates these operations, computational operations, to the sampler. And then the sampler, as I was saying, can be the one from NutPy, which is in Rust and Numba. And otherwise, it can be the one from NumPyro.
actually, you can call the NumPyro sampler with a PIMC model. And it's just super simple. Like in pm .sample, you're just like, there's a keyword argument that's nuts underscore sampler and you just say nutpy or NumPyro.
And I tend to use NumPyro a lot when I'm doing Gaussian processes because I don't know why, but so most of the time using nutpy, but when I'm doing Gaussian processes somewhere in the model, I tend to use NumPyro because like for some reason in their routine, in their algorithm, there is some efficiency they have in the way they compute the matrices. And GPs are basically huge matrices and dot products. so yeah, like usually NumPyRoll is usually very efficient for that.
And you can also use Jax now to sample your model. So we have like these different backends and it's enabled because PyTensor is that backend that nobody sees most of the time you're not implementing a Python or operation in your models. Sometimes we do that on PNC Labs when we're working on a very custom operation, but usually it's done under the hood for you.
And then Python compiles the graph, the symbolic graph, can dispatch that afterwards to whatever the best way of computing the posterior distribution afterwards is. Nice. You alluded there. to something that I've been meaning to get to asking you about, which is the Pi MC labs team. So you have Pi MC, the open source library that anybody listening can download.
And of course I haven't shown us for people to download and they can get rolling on doing their Bayesian stats right now, whether they're, it's already something they have expertise in or not. Hi, MC labs. It sounds like you're responsible and just fill us in, but I'm kind of, gathering that the team there is responsible both for developing. IMC, but also for consulting because you kind of you mentioned there, you know, sometimes we might do some kind of custom implementation.
So first of all, yeah, tell us a little bit about PI MC labs. And then it'd be really interesting to hear one or more interesting examples of how Bayesian statistics allows some client or some use case, allows them to do something that they wouldn't be able to do with another approach. Yeah. So yeah, first, go install PyMC on GitHub and open PRs and stuff like that. We always love that. And second, yeah, exactly.
PyMC is kind of an offspring of PyMC in the sense that everybody on the team is a PyMC developer. So we contribute to PyMC. This is open source. This is free. This is free and always will be as it goes. But then on top of that, we do consulting. what's that about? Well, most of the time, these are clients who want to do something with PMC or even more general with patient statistics.
And they know we do that and they do not know how to do that either because they don't have the time or to train themselves or they don't want to, or they don't have the money to hire a Bayesian modeler full time, various reasons. But basically, yeah, like they are stuck in at some point in the modeling workflow, they are stuck. It can be at the very beginning. It can be, well, I've tried a bunch of stuff. I can't make the model converge and I don't know why.
So it can be like a very wide array of situations. Most of the time people know. us because like me for the podcast or for PMC, most of the other guys for PMC or for other technical writing that they do around. So basically that's like, that's not really a real company, but just a bunch of nerds if you want. But no, that's a real company, but we like to define us as a bunch of nerds because like that's how it really started.
And it, in a sense of you guys actually consulting with companies and making an impact in that sense, it is certainly a company. Yeah. So yeah. So tell us a bit about projects. mean, you don't need to go into detail with client names or whatever, if that's inappropriate, but it would be interesting to hear some examples of use cases, use cases of Bayesian statistics in the wild, enabling capabilities that other kinds of modeling approaches wouldn't. Yeah. Yeah. Yeah. No, definitely.
Yeah, so of course I cannot enter into the details, but I can definitely give you some ideas. When I can actually enter into the details is a project we did for an NGO in Estonia, where they were getting polling data. So every month they do a poll of Estonian citizens about various questions. These can be horse -pull. races, horse races, polls. but this can be also, you know, news questions like, do you think Estonia should ramp up the number of soldiers at the border with Russia?
do you think, same sex marriage should be legal? Things like that. I hear some Overton window coming on. No, that's what I thought. I thought we might go there. Yeah, this is now I'm completely taking you off on a sidetrack, but Serge Macisse, our researcher came up with a great question for you because you had Alan Downey on your show. He's an incredible guest. I absolutely loved having him on our program. So he was on here in episode number 715.
And in that episode, we talked about the Overton window, which is related to what you're kind of just talking about. So kind of, you know, people What is, how does society think about say same sex marriage?
Where, know, if you looked a hundred years ago or a thousand years ago or 10 ,000 years ago or a thousand years into the future or 10 years into the future at each of those different time points, there's a completely, well, maybe not completely different, but there's a there's a varying range of people who think, you know, what's acceptable or what's not acceptable. And so, and this is You we were talking earlier in the episode about bias, so it kind of ties into this.
You, you know, you might have your idea as a listener to the show, you might be a scientist or an engineer and you think, I am unbiased, you know, I, know the real thing and, but you don't because you are a product of your times. And the Overton window is kind of a way of describing this on any given issue. There is some range. And it would fit a probability distribution where, you know, there's some people on a far extreme one way and some people on a far extreme the other way.
But in general, all of society is moving in one direction, typically in a liberal direction on a given social issue. And this varies by region. It varies by age. Anyway, I think Overton windows are really fascinating. and, yeah. so Completely derailed your conversation, but I have feeling you're going to have something interesting to say. Yeah, no, mean, that's related to that for sure.
yeah, basically, and that's really because yeah, we like I had also Alan Downey on the show for his latest book. And that was also definitely about that. So probably overthinking it was the book. Yeah. Yeah. Great. Great. Great. And yeah, great book. And so basically they like the, NGO, have, they had these survey data, right? And they're like, but there are the clients have questions and their clients are usually media or politicians.
it's like, yeah, but I'd like to know on a geographical basis, you know, like in these electoral districts, what do people think about that? Or in these electoral district, female educated. of that age, what do they think about same -sex marriage? That's hard to do because polling at that scale is almost impossible. It costs a ton of money. Also, polling is harder and harder because people answer less and less to polls.
At the same time, the polling data becomes less available and less reliable, but you have people who get more interested in what the polls have to say. It's hard. There is a great method to do that. What we did for them is come up with a hierarchical model of the population because hierarchical models allow you to share information between groups. Here the groups could be the age groups, for instance.
Basically, knowing something what a hierarchical model says, is, well, age groups are different, but they are not infinitely different. So learning about what someone aged 16 to 24 thinks about same -sex marriage actually already tells you something about what someone aged 25 to 34 thinks about that. And the degree of similarity between these responses is estimated by the model. So these models are extremely powerful. I love them. I teach them a lot.
And actually in the advanced regression course, the last lesson is all about hierarchical models. And I actually walk you through a simplified version of the model we did at Pintsy Labs for that NGO called SONC in Estonia. So it's like a model that's used in industry for real. you learn that. That's a hard model, but that's a real model. Then once you've done that, you do something that's called post stratification.
And post stratification is basically a way of debiasing your estimates, your predictions from the model, and you use census data to do that. So you need good data and you need census data. But if you have good census data, then you're going to be able to basically reweight the predictions from your model.
And that way, if you combine post -certification and hierarchical model, you're going to be able to give actually good estimates of what females, educated, age 25, 34 in this electoral district think about that issue. And when I say good, I want to say that it's like the confidence intervals are not going to be ridiculous. It's not going to tell you Well, these population think is opposed to gay marriage with a probability of 20 to 80%, which just covers basically everything.
So that's not very actionable. No, the model has like, it's more uncertain, of course, but it has a really good way of giving you something actually actionable. So that was a big project. I can dive into some others if you want, but that... That takes some, I don't want to deride the interview. That's great and highly illustrative. It gives that sense of how with a Bayesian model, you can be so specific about how different parts of the data interrelate.
So in this case, for example, you're describing having different demographic groups that have some commonality, like all the women, but different age groups of women as a sub node, as sub nodes of women in general. That way you're able to use the data from each of the subgroups to influence your higher level group.
And actually something that might be interesting to you, Alex, is that my introduction to both our programming and I guess, well, to hierarchical modeling is Gelman and Hill's book, which yeah, obviously Andrew Gelman you've already talked about on the show. Jennifer Hill also brilliant causal modeler and has also been on the Super Data Science podcast. And that was episode number 607.
Anyway, we're getting into lots of, there's lots of listening for people to do out there between your show and mine based on guests that we've talked about on the program. Hopefully lots of people with long commutes. So yeah, fantastic. That's a great example. Alex. Another library, open source library, in addition to PyMC that you've developed is Rviz, which has nothing to do with the programming language R. So it's A -R -V -I -Z or Zed, Rviz.
And this is for post -modeling workflows in VasionStats. So tell us about why, you know, what is post -modeling workflows? What does that matter? And how does Rviz solve problems for us there? Yeah, yeah, great questions. And I'll make sure to also, before it related to your previous question, send you some links with other projects that could be interesting to people like MediaMix Models. I've interviewed Luciano Paz on the show.
We've worked with HelloFresh, for instance, to come up with a MediaMix Marketing Model for them. Luciano talks about that in that episode. Also send you a blog post with spatial data, with Gaussian processes. That's something we've done for an agricultural client. And I already sent you a link of a video webinar we did with that NGO, with that client in Estonia. And we talked, we go a bit deeper into the project.
And I'll send you of course also the... the Learn Based Stats episode because the president, Thermo, the president of that NGO was on the show. Nice. Yeah. I'll be sure, of course, to include all of those links in the show notes. Yeah. Yeah. Because I guess people come from different backgrounds and so someone is going to be more interested in marketing, another one more in social science, another one more in spatial data. So that way people can pick and choose what they are most curious about.
So obvious. Yeah. What is it? That's basically your friend for any post -model, post -sampling graph. And why is that important? Because actually models steal the show and they are the star on the show. But a model is just one part of what we call the Bayesian workflow. And the Bayesian workflow just has one step, which is the modeling. But all the other steps don't have to do anything with the model. There is a lot of steps before sampling the model and then there is a lot of steps afterwards.
And I would argue that these steps afterwards are almost as important as the model. Why? Because it's what's going to face the customer of the model. Your model is going to be consumed by people who most of the time don't know about models and also often don't care about models. That's a shame because I love models, but you know, lots of the time they don't really care about the model. They care about the results.
And so a big part of your job as the modeler is to be able to convey that information in a way that someone who is not a stat person, a math person can understand and use in their work. Whether that is a football coach. or a data scientist or someone working in HelloFresh marketing department, you have to adapt the way you talk to those people and the way you present the results of the model. And the way you do that is with amazing graphs.
So a lot of your time as a modeler is spent thinking out how to decipher what the model can tell, what the model cannot tell. also very important, and with which confidence, and since we're humans, we use our eyes a lot, and the way to convey that is with plots. And so you spend a lot of time plotting stuff as a Bayesian modeler, especially because Bayesian models don't give you, you one -point estimate. They give you full distributions for all the parameters.
So you get distributions all the way down. So, That's a bit more complex to wrap your head around at the beginning, but once your brain is used to that gym, that's really cool because that gives you opportunities for amazing plots. yeah, like RVs is here for you for that. It has a lot of the plots that we use all the time in the Bayesian workflow. One, to diagnose your model. So to understand if there is any red flag in the convergence of the model.
And then once you're sure about the quality of your results, then how do you present that to the customer of the model? then, then obvious also has a lot of plots for you here. And the cool thing of obvious is that it's platform agnostic. What do I mean by that? It's that you can run your model in PIMC in a pyro in Stan, and then use obvious because obvious is expecting a special format of data that all these PPLs can give you, which is called the inference data object.
Once you have that, ARVIS doesn't care where the model was run. And that's super cool. And also it's available in Julia. So that's Python package, but there is the Julia equivalent for people who use Julia. So yeah, it's a very good way of starting that part of the workflow, which is extremely important. Nice. That was a great tour. And of course, I will again have a link to ARVIS in the show notes for people who want to be using that for your post modeling needs with your vision models.
including diagnostics like looking for red flags and being able to visualize results and passes off to whoever the end client is. I think it might be in the same panel discussion with the head of that NGO, Tormo Uristo. Yes. That's my Spanish. I'm in Argentina right now, so the Spanish is automatic. Actually, I'm relieved to know that you're in Argentina because I was worried that I was keeping you up way too late.
No, no, no, no, no. Nice. So yeah, so in that interview, Tarmu talks about adding components like Gaussian processes to make models, Bayesian models, time aware. What does that mean? And what are the advantages and potential pitfalls of incorporating advanced features like time awareness into Bayesian models? Yeah, great research, can see that. Great research, Serge Pessis, really. Yeah. No, that's impressive. I had a call with the people from Google Gemini today.
So they're very much near the cutting edge of developing Google Gemini alongside Cloud 3 for Anthropic and of course, GPT -4, GPT -4 -0, whatever, from OpenAI. These are the frontier of LLMs. So I'm on a call with... half a dozen people from the Google Gemini team. And they were insinuating kind of near the end with some of the new capabilities they have. And there are some cool things in there, which I need to spend more time playing around with like gems.
I don't know if you've seen this, but the gems in Google Gemini, they allow you to have a context for different kinds of tasks. like, for example, there are some parts of my podcast production workflow where I have different context, different needs. at each of those steps. And so it's very helpful with these Google Gemini gems to be able to just click on that and be like, okay, now I'm in this kind of context. I'm expecting the, the LLM to output in this particular way.
And the Google Gemini people said, well, and maybe you'll be able to use these gems to kind of be replacing, you know, within the workflow of other people working on your podcast, you'll be able to use them to replace. I was like, you know, you know, for example, they give the example of research and I was like, I hope that our researcher, for example, is using generative AI tools to assist his work. But I think we're quite a ways away with all of the amazing things that alums can do.
I think we're still quite a ways from like the kind of quality of research that search mrs can do for this show. Yeah, yeah, yeah. We're still a ways away. Yeah, yeah, no, no, for sure. But that sounds like fun. Yeah. Anyway, sorry, I derailed you again. Time warrants. Yeah, Indeed. And then I love that question because I love GPs. So thanks a lot. And that was not at all a setup for the audience, processes, GPs, Yeah, I love Gaussian processes.
And actually just sent you also a blog post we have on the PrimeCLabs website by Luciano Paz about how to use Gaussian processes with spatial data. So why am I... telling you that because Gaussian processes are awesome because they are extremely versatile. It's what's called a non -parametric that allows you to do non -parametric models. What does that mean?
It means that instead of having, for instance, a linear regression where you have a functional form that you're telling the model, I expect the relationship between X and Y to be of a linear form. y equals a plus b times x. Now, what the Gaussian process is saying is, I don't know the functional for between x and y. I want you to discover it for me. So that's one level up, if you want, in the abstraction. And so that's saying y equals f of x. Find which x it is.
So you don't want to do that all the time because that's very hard. And actually, you need to use quite a lot of domain knowledge on some of the parameters of the GPs. But I want to turn to the details here, but I'll give you some links for the show notes. But something that's very interesting to apply GPs on is, well, spatial data, as I just mentioned, because you don't really know in a plot, for instance, of, so not a plot graph, but a plot, a field plot.
There are some interactions between where you are in the plot and the crops that you're going to plant on there. But you don't really know what those interactions are. Like it interacts with the weather also with a lot of things. And you don't really know what the functional form of that is.
And so that's where GP here is going to be extremely interesting because it's going to allow you to see in 2D, try and find out what these correlation between X and Y are and take that into account in your model. That's very abstract, but I'm going to link afterwards to a tutorial. We actually just released today in PIMC a tutorial that I've been working on with Bill Engels, who is also a GP expert. And I've been working with him on this tutorial for an approximation, a new approximation of GPs.
And I'll get back to that in a few minutes. But first, why GPs in time? So you can apply GPs on spatial data on space. But then you can also apply GPs on time. Time is 1D most of the time, one dimensional. Space is usually 2D. And you can actually do GPs in 3D. You can do spatial temporal GPs. That's it. That's even more complicated. But like 1D GPs, that's really awesome. Because then most of the time when you have a time dependency, it's non -linear.
For instance, that could be the way the performance of a baseball player evolve within the season. You can definitely see the performance of a baseball player fluctuate with time during the season. And that would be nonlinear, very probably. the thing is you don't know what the form of that function is. And so that's what the GP is here for. It's going to come and try to discover what the functional form is for you.
And that's why I found GPs extremely, like they are really magical mathematical beasts. First, they're really like beautiful mathematically. And a lot of things are actually a special case of GPs, like neural networks. Neural networks are actually Gaussian processes, but a special case of Gaussian processes. Gaussian random walk. They are a special case of Gaussian processes. So they are a very beautiful mathematical object, but also very practical.
Now, as Uncle Ben said, with great power comes great responsibility. And GPs are hard to yield. It's a powerful weapon, but it's hard to yield. It's like Excalibur. You have to be worthy to yield it. And so it takes training and time to... use them, but it's worth it. And so we use that with Thermo, Juristo from that Estonian NGO. But I use that almost all the time. Right now I'm working more and more on sports data and yeah, I'm actually working on some football data right now.
And well, you want to take into account these wheezing season effects from players. I don't know what the linear form is. And right now, the first model I did taking the time into account was just a linear trend. So it's just saying as time passes, you expect a linear change. So the change from one to two is going to be the same one than the one from nine to 10. But usually it's not the case with time. It's very non -linear. And so here, you definitely want to apply a GP on that.
You could apply other stuff like random walks. autoregressive stuff and so on. I don't personally don't really like those models. find them like, it's like you have to apply that structure to the model, but at the same time, they're not that easier to use than GPs. So, know, might as well use a GP. And I'll end this very long answer with a third point, which is now it's actually easier to use GPs. because there is this new decomposition of GPs that's Hilbert space decomposition. So HSGP.
And that's basically a decomposition of GPs that's like a dot product. So kind of a linear regression, but that gives you a GP. And that's amazing because GPs are known to be extremely slow to sample because it's a lot of matrix multiplication, as I was saying. at some point. But with HSGP, it becomes way, way faster and way more efficient. Now, you cannot always use HSGP, there are caveats and so on. But Bill and I have been working on this tutorial. It's going to be in two parts.
The first part was out today. And I'm going to send you the links for the show notes here in the chat we have. That's up on the PMC website. If you go, that's HSGP First Steps and Reference. And we go through why you would use HSGP, how you would use it in PMC, and the basic use cases. And the second part is going to be the more advanced use cases. Bill and I have started working on that, but it's always taking time to develop good content. on that front.
yeah, we're getting there and it's open source, so we're doing that on our free time unpaid. that always takes a bit more time. But we'll get there. And finally, another resource that I think your listeners are going to appreciate is I'm doing a webinar series on HSGP where we have a modeler that who comes on the show and shares our screen and does live coding. so the first part is out already. I'm going to send you that for the show notes.
had Juan Orduz on the show and he went into like, the first part of how to do HSGPs and what are even HSGPs from a mathematical point of view because Juan is a mathematician. So yeah, like I'll end by my very long, passionate rant about about GPS here, but long story short, GPS are amazing. And that's a good investment of your time to be skillful with GPS. Fantastic. Another area that I would love to be able to dig deep into.
And so our lucky listeners out there who have the time will now be able to dig into that resource and many of the others that you have. suggested in this episode, which we've all got for you in the show notes. Thank you so much. Alex, this has been an amazing episode. Before I let my guests go, I always ask for book recommendation and you've had some already for us in this episode, but I wonder if there's anything else.
you had, the recommendation you already had was Bernoulli is something about Bernoulli. Right. Bernoulli's fallacy, the crises, the crisis of modern science and the logic of science, think. yeah. Yeah, I'll send you that actually, the episode with Aubrey and David Spiegelhalter, because these are really good, especially for less technical people, but who are curious about science and how that works. think it's very good entry point. Yeah, so this book is amazing.
my God, this is an extremely hard question. I love so many books and I read so many books that I'm taken aback. So books, I would say books which... I find extremely good and have influenced me because a book is also like, it's not only the book, right? It's also the moment when you read the book. So like, yeah, if you like a book and you come back to it later, you'll have a different experience because you are a different person and you have different skills.
And yeah, so I'm going to cheat and give you several recommendations because I have too many of them. so technical books, I would say The Logic of Science by E .T. Jains. E .T. Jains is an old mathematician scientist, it's like in the Bayesian world, E .T. Jains is like a rock star. Definitely recommend his masterpiece, The Logic of Science. That's a technical book, but that's actually a very readable book. And that's also very epistemological. So that one is awesome.
Much more applied if you want to learn. patient stats. A great book to do that, Statistical Rethinking by Richard Michael Rath. Really great book. I've read it several times. Any book by Andrew Gellman, as you were saying, definitely recommend them. They tend to be a bit more advanced. If you want a really beginner's one, his last one, actually Active Statistics, he's a really good one. I just had him on the show, episode 106. John? for people who like numbers, say like that.
And I remember that when I was studying political science, Barack Obama's book from before he was president, I don't remember the name. I think it's the Audacity of Hope, I'm not sure. But his first book before he became president, that was actually a very interesting. Dreams from my father? Yes. Yeah, yeah, this one. Dreams from my father. Yeah, yeah, yeah. Very interesting one. The other ones were a bit more political. I found a bit less interesting. But this one was really interesting to me.
And another one for people who are very nerdy. So I'm a very nerdy person. I love going to the gym, for instance. I have my, I can do my own training plan, my own nutritional plan. I've dug into that research. I love that. because I love sports also. Another very good book I definitely recommend to develop good habits is Kathy Milkman's How to Change, the Science of Getting from Where You Are to Where You Wanna Go. Extremely good book full of very practical tips.
Yeah, that's an extremely good one. And then a last one that I read recently. no, actually two last ones. All right. Yeah. One last two. Penultimate, I think you say. How Minds Change by David McCraney for people who are interested in how beliefs are formed. Extremely interesting. he's a journalist. He's got a fantastic podcast that's called You're Not So Smart. And I definitely recommend that one. And yeah, that's how people change their mind, basically, because I'm very interested in that.
And in the end, is a, like, it's a trove of wisdom, this book. And very, like, last one, Promise. I also am extremely passionate about Stoicism, Stoic philosophy. And that's a philosophy I find extremely helpful to live my life and navigate the difficulties that we all have in life. And a very iconic book in these is Meditations from Marcus Aurelius, reading the thoughts of a Roman emperor, one of the best Roman emperor there was.
It's really fascinating because he didn't treat that to be published. It was his journal, basically. That's absolutely fascinating to read that and to see that they kind of had the same issues we still have. You know, so that's yeah, fantastic. It's my, I read it very often. Yeah. I haven't actually read meditations, but I read Ryan holidays, the daily stoic. Yeah. yeah. Yeah. Yeah. And that's really good.
it's, yeah, 366 daily meditations on wisdom, perseverance, and the art of living and based on Stoke philosophy. And so there is a lot from Marcus or really soon there. He's probably the plurality of content and wow, it is. It is mind blowing to me how somebody to millennia ago is the same as me. mean, I mean, that's holding myself in. I'm not a Roman emperor. the things I write will not be studied 2000 years from now.
But nevertheless, the connection you feel with this individual from 2000 years ago and the problems that he's facing and how similar they are. the problems that I face every day, it's staggering. Yeah. Yeah. Yeah. No, that's incredible. Me, something that really talked to me was, well, that I remember is at some point he's saying to himself that it's no use to go to the countryside to escape everything because the real retreat is in yourself.
It's like, if you're not able to be calm and find equanimity in your daily life. It's not because you're going to get away from the city and Rome was like the megalopolis at the time. It's not because you get away from the city that you're going to find tranquility over there. You have to find tranquility inside. then, yeah, you'll go to the country that is going to be even more awesome. But it's not because you go there that you find tranquility.
And that was super interesting to me because I was like, wait, because I definitely feel that when I'm in a big, big metropolis. At some point, I want to get away. But I was like, wait, we're leaving that already at that time where they didn't have internet, they didn't have cars and so on. But for them, it was already something that was too many people, too much noise. I found that super interesting. For sure. Wild. Well, this has been an amazing episode, Alex.
I really am glad that Doug suggested you for the show because... This has been fantastic. I've really enjoyed every minute of this. I wish it could go on forever, but sadly all good things must come to an end. And so before I let you go, very last thing is do you have other places that we should be following you? We're going to have a library of links in the show notes for this episode. And of course we know about your podcast, learn Bayesian statistics. We've got the intuitive Bayes.
educational platform and open source libraries like PyMC and RViz. In addition to those, is there any other social media platform or other way that people should be following you or getting in touch with you after the program? Well, yeah, thanks for mentioning that. So yeah, intuitive base, learning vision statistics, PyMC Labs, you mentioned them. And I'm always available on Twitter, alex underscore andora. like the country and that's where it comes from. Because it has two Rs and not only one.
And so when I say it in another language than Spanish, then people write it with just one R. otherwise, LinkedIn also, I'm over there. So you can always reach out to me over there, LinkedIn or Twitter. also, yes, me podcast suggestions, stuff like that. always... the lookout for something cool. again, yeah, thanks a lot, for having me on. Thanks a lot, Doug, for the recommendation. yeah, that was a blast. I enjoyed it a lot. So thank you so much.
This has been another episode of Learning Bayesian Statistics. Be sure to rate, review, and follow the show on your favorite podcatcher, and visit learnbaystats .com for more resources about today's topics, as well as access to more episodes to help you reach true Bayesian state of mind. That's learnbaystats .com. Our theme music is Good Bayesian by Baba Brinkman. Fit MC Lance and Meghiraan. Check out his awesome work at bababrinkman .com. I'm your host. Alex Andorra.
can follow me on Twitter at Alex underscore Andorra, like the country. You can support the show and unlock exclusive benefits by visiting Patreon .com slash LearnBasedDance. Thank you so much for listening and for your support. You're truly a good Bayesian. Change your predictions after taking information in. And if you're thinking I'll be less than amazing, let's adjust those expectations.
Let me show you how to be a good Bayesian Change calculations after taking fresh data in Those predictions that your brain is making Let's get them on a solid foundation