The Great Basilisk

My name is Robert Lamb, and I'm Joe McCormick. And since it's October, we are of course still exploring monsters, terrifying ideas and so forth, and boy, have we got one for you today. I just want to issue a warning right at the beginning here that today's episode is going to concern something that a few people would consider a genuine information hazard, as in an idea you that is itself actually dangerous. Now I I don't, having looked

into it, I don't think that is the case. I don't think this episode will hurt you, but just a warning if you think you're susceptible to terror or nightmares or something when presented with a thought experiment or the possibility of being, say, sent to a literal hell created by technology, and you think that idea could infect you, could make you afraid. This might not be the episode for you, right, But then again, I assume you're a listener to stuff to blow your mind. You've probably already

encountered some thought hazards on here. You've survived those. Generally speaking,

I have faith in you to survive this one. However, if you are going to take either of our warnings seriously, I will let you know that the first section of this podcast is going to deal with the mythical basilisk, the folkloric basilisk, and some of the you know, the monstrous fund to be had there before we explore the idea of Rocco's Basilisk, and in that we're gonna be talking about this, uh, this idea that emerges where technological singularity,

navel gazing, thought experimentation, a little that dash of creepy pasta, and some good old fifth fashion supernatural thinking all converge into this kind of nightmare scenario. Now, as we said, this idea is believed by some to be a genuinely dangerous idea and that even learning about it could put you at some kind of risk. I think there are strong reasons to believe that this is not the case, and that thinking about this idea will not put you

at risk. But again, if you're concerned, you should stop listening now, or stop listening after we stopped talking about the mythical basilisk now I have I just want to say at the beginning, listeners have suggested us talk about Rocco's basilisk before this idea that is at least purportedly an information hazard, a dangerous idea, and I've I've hesitated to do it before, and not because I think it's particularly plausible, but just because, you know, I wonder, what

is the level of risk could that you should tolerate when propagating an idea. If you think an idea is unlikely but maybe has a zero point zero zero zero zero one percent chance of causing enormous harm to the person you tell it to, should you say the idea or not? I don't know. I feel like people generally don't exercise that kind of caution when they're like sharing links with you, Like sometimes they'll be like, like not

safe for work. But but then you click on it anyway, and then sometimes you're like, oh, I wish I had not seen that, or I wish I had not read that, and now that's in my head. Now that's in my head forever. Well. One of the problems with this idea is whatever you think about whether or not you should discuss ideas that may be dangerous to hear in some extremely unlikely off chance. Uh. Part of the problem is what happens when those ideas are just already set loose

in in society. I mean now people on television shows and all over the internet or talking about this idea. There are a bunch of articles out about it. So it's not like you can keep the cat in the bag at this point, right this Roco's basilist has already been a a gag point on the hit HBO show Silicon Valley, which is a fabulous show, and I love the way that they treated Rocco's basilisk on it. But uh, yeah, if they're covering it, there's no danger in us covering

it too. That's the way I look at it, right, And at least I would hope that the way we cover it can give you some reasons to think you should not be afraid of digital hell, and also to think about the general class of what should be done about something that could have in fact been a real information hazard in some other case. So that's all our whole preamble for before we get to that section. But before we get to that section, we're gonna be talking

about basilisks today. Boy, is the basilisk a great monster? Yes, also known as the basil cock, the basil cock, the basil e cock, basically any version of that of basil and cock that you can put together, um, that you can slam together. Then it has been referred to as such at some point in its history. Now, a lot of people I think probably encountered a version of the basilisk from Harry Potter. But Robert, I know that was not your entryway, right, I encountered it for the first time.

I believe in dungeons and dragons, of course, because it's a it's a multi legged reptile with a petrifying gaze. Um say that again, multi legged reptile with a petrifying gaze, petrifying gas to turn you to stone. Yeah, and I is ever over call correctly. It has some cool biology, where like the the it turns you to stone and then like bust you into pieces. Then it eats the stone pieces, but then its stomach turns the stone back

into flesh. And so if you get like the stomach juices from a basilisk, then you can use it to undo petrification spells, that sort of thing. It's a lot of fun, arguably more fun than than the basilisk is at times in folklore tradition, because one of the things is, if you you like me, you didn't grow up hearing about the basilisk. Part of it is because there are no there aren't really any great stories about the basilisk. Slaying the basilisk was no heroes, great ordeal. Oh yeah,

I at least have not come across that. Yeah, it really helps, like if like the hydra, I mean, the hydra is arguably so much cooler, and then also it's one of the labors of Hercules, and there's a cool story about how they defeat it. Well, maybe it's because there is no way to defeat the basilisks short of say, weasel of fluvium, which we will get to. Yeah, don't don't give it away, I'm sorry. Should we should we edit that out. No, we should leave it. It's a

it's it's a thought hazard. Any basilists listening. Still, there's so much more to the basilist than just this cool DND creature, because it's not just a monster. It's not just something you encounter in the dungeon. It is it is a king. Oh, I see, now, I know you've made note of the fact that Borges mentions the basilisk and his Book of Imaginary Beings. He does now he he translated translated his meaning little king, little king, which

I like when I was reading. And Carol Rose she's She points out that the name stems from the Greek basilius, which means king, so king or little king. I tend to like the little king translation because I feel like it it ties in better with what we're going to discuss, well, the ancient BEASTI area ideas of the basil I believe to say that it's not that big, right, It's pretty small. Yeah. Yeah. Now.

The keen part, though, refers to a crest or, a crown like protusion that is on the creature's head, and in some depictions it's no mirror biological ornament, but an actual regal crown. It means something. Yeah. Now, the descriptions very greatly, and it emerges largely from European and Middle Eastern legend and folklore from ancient times to roughly the seventeenth century, and that's when the basilists became less popular.

In the earlier descriptions, though, it is indeed small, and it's just a grass snake, only it has a crown like crest on its head and has this weird practice of floating above the ground like vertically erect. That's creepy, of course, and then again that that does play on sometimes when you see snakes rise up out of a coil,

it can be startling how high they rise. Yeah, I mean, if I feel like I've grown up seeing images and videos of cobra's doing their their dance, so I've kind of I've kind of lost any kind of appreciation for how bizarre that is to look at. You know, if you're used to seeing a snake slither, to see it stand up and uh, you know and rare back and and and flare its hood. Absolutely. Yeah. So the basilisk is said to be the king of the reptiles, but you know, don't be so foolish is to think that

only it's bite is lethal, like some of our venomous snakes. Now, every aspect of the basilisk is said to just reek of venom and death, every every aspect. If you touch it, if you inhale its breath, if you gaze upon it at all, then you will die. Wait, what about its saliva? Yep, saliva, blood smell, gaze? Presumably I didn't see any reference that the presumably it's it's urine, it's excrement. I mean, it's excrement has to be poisonous, the excrement of a basilisk.

It sounds absolutely deadly. Wouldn't it be a great inversion if it's excrement was the only good part about it? And maybe so that can heal your warts? Yeah? And uh. One thing that Carol Rose pointed out in in her her entry about the basilisk and her one of her monster Encyclopedia's, she said that when it's not killing everything in its path, just via that you know the audacity of its existence, it would actually spit them at birds flying overhead and bring them down to eat them, or

just out of spite. I get the idea. Just had a spite. You know, it's just just it's just my ful death, that's all it is. Okay. So where do I find a basilisk? Well, in the desert, of course. But it's more, it's more accurate to say that the desert is not merely the place where it lives, but it is the place that it makes by living. Like everything in its path dies, and therefore the desert is

the result of the basilisk. And there's a there's actually a wonderful um a description of the basilisk that comes to us from Plenty of the Elder in his The net Real History Man. We've been hitting Plenty a lot lately. I guess we've been talking about monsters. Huh. If're talking about monsters, especially ancient monsters, that you know, he's he's one of the great sources to turn to. So Joe,

would you read to us from the natural History? Oh? Absolutely, there is the same power also in the serpent called the basilisk. It is produced in the province of Syrene, which that is the area to the west of Egypt. It's like Libya. I think there's a settlement known as like Syrene, Syrene being not more than twelve fingers in length. Is that fingers long ways or fingers sideways? Oh, either way you cut it, it's not a huge creature. It has a white spot on the head, strongly resembling a

sort of diadem. When it hisses, all the other serpents fly from it, and it does not advance its body like the others by a succession of folds, but moves along upright and erect upon the middle. It destroys all shrubs, not only by its contact, but those even that it has breathed upon. It burns up all the grass too, and breaks the stones. So tremendous is its noxious influence.

It was formerly a general belief that if a man on horseback killed one of these animals with a spear, the poison would run up the weapon and kill not only the rider but the horse as well. Oh man, I love that. So it's though the blood is it's like that like a xenomorph's blood, right or or kind of like It reminds me to of Grendel's blood that was said to like melt the weapon that they wolf

used against it. Oh, but it's worse than that. It doesn't just get the weapon, It gets the person holding the weapon and the horse that that person is touching. I know, it feels unfair that the horse is roped into this as well. Yeah, the horse didn't even sign up for going to fight a basilisk. It's just trying to get some oats. But furthermore, what is this horse rider doing out in the waste land of Syrene trying

to kill a basilisk? Well, lesson learned. Lesson learned now that the basilist becomes a popular creature, and even though the basil usk itself is it doesn't seem to have been mentioned in the Bible, it ends up being like roped into it via translations. Oh yeah, it's kind of

like the unicorn. Actually, yeah, exactly. Yeah. We discussed in our episode on unicorns how the were their words in the Bible that have been translated saying the King James translation of the Bible into unicorn because the translators didn't know what the word referred to. We think now that maybe the word probably referred to the ROCs, an extinct bovine creature that once lived around the ancient Mediterranean. Yeah, so you see the basilist pop pop pop up in

certain translations of the Book of Jeremiah. The Book of Psalms, Uh what it's associated with the devil or evil, and nothing short of the coming of the Messiah can can hope to end its rule. While well, have you got a quote for me? Yes, there's one translation of Psalms. This is the Brinton Septuagint translation quote thou shalt tread on the asp and basilisk and al shout trample on the lion and dragon. Now European beast areas of the

eleventh and twelfth century. They both mostly maintained Plenty's description, but then they described a larger body. They began to Essentially, the monster began to grow. We got to beat this thing up here. Yeah, it ended up having spots and stripes and a few other features were thrown in. Um fiery breath, a bellow that kills. Well, that only makes sense if every other thing about it kills. Yea, it makes noise. That should kill things too. Also, the ability

to induce hydrophobia madness. I found that interesting because this clearly has to be a reference to the actual hydrophobia that is inherent in rabies. Yeah, the idea there being that, and I think later stages of Araby's infection. Persons will often have difficulty swallowing, and so that they they're said to refuse drinking water. So Plenty has some additional information

here about how you might deal with the pass list. Okay, so I assume not ride up on a horse and stab it right, Well, tell me what it is to this dreadful monster. The effluvium of the weasel is fatal, a thing that has been tried with success, for kings have often desired to see its body when killed. So true is it that it has pleased nature that there should be nothing without its antidote. The animal is thrown into the whole of the basilisk, which is easily known

from the soil around it being infected. The weasel destroys the basilisk by its odor, but dies itself in the struggle of nature against its own self. And John Bostock, who provided the translation of this, he adds that there's probably no foundation for this account of the action of the effluvium of the weasel upon the basilisk or any other species of serpent. But this is letting us know that throwing a weasel in there to bleed on it or secrete fluids or whatever. That's not going to kill

this mythical monster. But this is interesting though, because weasels, especially the stout, we're thought to be venomous um. And it's worth noting that we do have some venomous mammals in the natural world, such as various shrews, and even the slow loris, the only known venomous primate. I don't think I knew that the slow loris was venomous. Throw it into a hole with a basilisk and and I'm I'm betting on the loris. But anyway, BEASTI areas of

the time. They presented a few different ways that you could kill the bassilist. So the weasels one, weasels one always carry a weasel. Also, this one's a little more elegant, but I have a crystal globe with you to reflect its own petrifying gaze back upon the basilisk. Oh, so it's like Perseus and Medusa exactly mirror. Yeah, basically they just stole the idea from Medusa here, but then also carry with you a cockrell or a young rooster. The

basilist will becoming enraged by the bird's crown. The idea that this bird has a crown as well, and the basilist will die from a lethal fit. That's a jealous king. Yeah, I believe a similar thing occurs when someone refuses to believe dinosaurs had fetishes, you know, how dare the bird rise above the mighty reptile and then it just loses its mind and dies. We can only hope the producers

of the Jurassic World movies avoid this fate. So you see the basketlists show up in a number of different writings. It's just kind of a common um, really a symbol and idea that can be employed. Uh. And they even to see it show up in the Parson's Tale in the in Jeffrey Chaucer in the Canterbury Tales. Yes, quote, these are the other five fingers which the devil uses

to draw people towards him. The first is the lecterous glance of a foolish woman or a foolish man, a glance that kills just as the basilists kills people just by looking at them. For the covetous glance reflects the intentions of the heart. You know, this kind of thing is actually one of my one of my favorite things about monsters, especially ancient medieval monsters and uh and so forth. Is that they often aren't just like a large, dangerous animal,

but they embody some kind of value. They represent something, They give you something to compare other things too, like they they they're very useful as a metaphor uh. And then really we see some more thing with the bassilisk.

It becomes less far less the situation where people are like, hey, you need to be careful because there's a basilisk in the desert, and more and more just a useful model, useful, ridiculous idea that we use to illustrate something that is presumably true about the world and then ultimately loses all meaning and just winds up on, you know, a heraldry

and decorations. Now, as we've seen already, the the basilisk has been through some transformations of form, and I assume those transformations must have somewhat continued as time goes by, and it becomes uh, it transforms into this idea of the cock of thrace. This uh, this rooster with a curling serpent's tail. In fact, if you if you go looking around for images the basilist, sometimes you will find this image instead. You really will find you'll find this

alongside all the other images. Um. So again a reptile to bird transfer transformation that just must enrage those who oppose feathered dinosaurs. And it does feel like a shame because we have this vile reptile. It becomes the kind of a weirdo bird instead. And it said that it's it's it's what happens when you have a seven year old chicken egg hatched by a toad who undead avians. Yeah,

but it's also made deadlier in these newer versions. So now it has that that poison blood power that Plenty describes. It also rots fruit and poisons water everywhere it goes, So it becomes this kind of embodiment of desolation and death, and the idea itself becomes popular. It influenced the naming of a tutor canon like literally a cannon that shoots called the basilisk, just because it's it's such a powerful weapon, we have to name it after this powerful, deadly monster. Uh.

And it eventually got some of its reptilian features back. Um. The artist Andrew Vandi has this excellent, excellent depiction of it in Natural History of Serpents and Dragons that gives it scales and these. It's like a fat, scaly reptile bird with eight rooster legs, which I just love. This will probably be the illustration for this episode on our on our website, But after that the creature largely became

just a part of European heraldry. It's just something you would see as a mere decoration or occasionally just a literary reference. Now, one thing that we want to be careful about is that we should not confuse the basilisk of legend the monster with true extent. Basilisk lizards also known sometimes as the Jesus Christ lizard or the Jesus lizard, for their ability to run across the surface of water without sinking for up to about four point five meters

or about fifteen feet. Uh. If you've ever seen video this, it's really cool. How how do they do that? I've often wondered. I didn't know until I looked it up for this episode. Apparently what they've got is big feet and the ability to run very fast. And what happens is when they run, they slap the water very hard with each down stroke of the foot. And it has to do with the way that the rapid motions of their feet create these air pockets around their feet as

they move. I was reading an article a New Scientist where some researchers who were working on this problem said that in order for an eight m or a hundred and seventy five pound human to do this, you would have to run at about a hundred and eight kilometers per hour about sixty seven miles per hour across the

surface of the water. But anyway, you can find basilisk lizards in South America and Central America and Mexico and uh as far as I know, they do not kill with a glance, and you cannot fight them with weasel fluvium. All right, Well, that pretty much wraps up the mythical, legendary, folkloric basilisk um it's rise and individual fall. But we're gonna take a break, and when we come back, we are going to get into this idea of of Rocco's Basiliska,

the great basilisk, the once and perhaps future king. Thank you, thank alright, we're back. Al right. So, as we mentioned earlier, we're about to start discussing an idea that has been classed by some as something that could be an information hazard. An idea that simply by thinking about it you somehow increase the chance of harm to yourself. So just another warning that again I don't think that's the case, but if that kind of thing scares you, then then perhaps

you can tune out. Now, all right, for those of you who decided to stick around, let's proceed. So we're talking about Rocco's Bassilist. This is an idea that goes back to around and it was proposed by a user at the blog less Wrong, by a user named Rocco

now less Wrong. I think it's a website that's a community that's associated with the rationalist movement somewhat, the rationalist movement being a movement that's concerned with trying to optimize thinking, like to eliminate bias and error, but especially among people who in this case are concerned with the possibilities of a technological singularity and what all that means and how how risks can be avoided, and of course what we've talked about this strain of thinking before you know, we

we've introduced I think some some skepticism about the idea of a technological singularity. I don't know fully yet how I come down on the dangers of a I debate, but I think it's at least something we're thinking about worth taking seriously. Yeah. I mean we've talked about, for instance, the work of of Max tech Mark and his arguments about how we need it. We need to be concerned about building the right kind of AI, and we need to we need we need to have serious discussions about it.

Not not mere you know sci fi um dreams regarding it or nightmares regarding it. You know, we need to we need to think seriously about how we're developing our technology. Yeah, we've talked about, say, the work of Nick Bostrom before, and criticisms by people like Jarren Lane in Air. Yes, but okay, give me the short version of the basilisk before we explain it a little more. Okay, So the idea here is that an AI superintelligence will emerge an

entity with just godlike technological powers. You know, it can you name and it can do it through it's it's it's technological power, it's interconnectedness. Basically, if it's physically possible, this computer can do it right, or will send a drone to do it or what have you. Uh So, Yeah, we've discussed this a bit in the podcast before, just the idea of you know, and then if you have this future king is it gonna be good or is it gonna be bad? Is it going to be a malevolent?

Is it going to be ruthless? And it's a and it's ascension. And that's the case with the basilist, the idea that it is ruthless, that you are either with it or you are against it, and it actually doesn't have to be malicious. It could actually even be well meaning.

It could just have ruthless tactics. Yes, yeah, that's that's that's also part of the argument is like, yeah, it wants to bring the best good for all humanity, but how it gets there, it'll do whatever it absolutely has to do it, such as you know, again, punishing anybody who stands against it, punishing even those who do not rise to support it um and that means demanding absolute devotion not only in its future kingdom, but in the

past that preceded it in our world as well. In other words, it will punish people today who are not actively helping it come into being tomorrow, and even those who have died, it is said, or choose death by their own hand. Rather than succumbed to the Great Basilist, will be resurrected as digital consciousness is and then tormented for all eternity. And it's dripping black cyber dungeon. Hell, the Great Bassilists, all hell, the Great Basilist, all hell,

the Great Basilists, all hell, the Great Basilisk. Well wait, what was that? I don't know? Um right, did you hear that? Okay, all right, we'll just keep going. So calling it a basilisk here, invoking the mythological bassilisk is is really a clever choice because it takes it one step further. Not only to look at the basilisk, but just to think of the basilisk is to uh invite death. Right. Merely to know about Rocco's Basilisk is enough to are too.

According to the model that's presented here, damn your digital soul to everlasting horror. And the only way to avoid such a fate then is to work in its favor, which, by the way, I think we're doing uh with this podcast, We're not well, I mean, I feel like we're giving lip service to the Great Basilisk, just in case, you know,

if the Great Basilist rises to power. Well, hey, we did that podcast and we even had a shirt that says all hell the Great, the Great Basilisk that's available on our t shirt store, So you know, we have we we have you know, our our options covered here. So that's the But that's the idea of the nutshell is that a future king ai king will rise and if you don't work to support it now knowing that it is going to exist, then you will be punished

for it. So one of the principles underlying the idea of Rocco Spasilisk is the idea of timeless decision theory, which, if you want a pretty simple, straightforward explanation of it, there is one in an article on Slate by David Auerbach called the most terrifying thought experiment of all time. This, by the way, I would say, I don't totally endorse everything our back says in that article. I mean, obviously that should be the case for any article we site.

But but he does at least have a pretty clear and easy to understand explanation of how this works. Or I don't know, what, would you agree, Robert that it's at least somewhat easy to understand? Oh? Yes, I would. Uh. There's another piece, by the way, by Beth Singler in Ian magazine called Faith. That's faith in lower case but

with the AI capitalized. But anyway, our box points out that that much, yeah, much of the thought experiment is based in the timeless decision theory t DT, developed by Less Wrong founder Elias Ar Yudkowski, based on the older thought experiment Newcomb's Paradox from the late sixties and early seventies, attributed to theoretical physicist William Newcom. Now you might be wondering who's this Yudkowski. Guys. Is he just some user on a random website I've never heard of before today?

Or is he like a name in his field? Uh? And he Uh, he has. He has a name of note. He is also the founder of the Machine Intelligence Research Institute, and his idea of working toward a friendly AI is touted by many, including Max tegmark I mentions it several times in his book Life three point oh, describing Yudkowski has quote an AI safety pioneer. Yeah, I mean in

in a weird way. He is a guy who posts on the internet, but he's a very influential one, especially among people who think about artificial intelligence Lawyeah, I mean, ultimately, what are any office but just people who post stuff on the internet post will one day be read by the great baskets. Okay, so we'll try to explain the

idea of timeless decision theory. So you start off with this idea of Newcome's paradox, right, right, And the paradox is essentially this, A super AI presents you with two boxes. One you're told contains a thousand dollars. That's box A. That's box A. Box B might contain one million dollars, or it might contain nothing, right, and you're left with two options here. These are the options that are given

to you. You can either pick both boxes, ensuring that you'll get at least one thousand dollars out of the deal, maybe that extra million two if it's in there. Or you can just pick pick box B, which means you could get a million dollars or you could have nothing. So uh, and I do want to add that just picking the thousand dollar box is not an option here,

because I was thinking about that too. Couldn't I just give the super AI the middle finger and say, I'm not playing your silly games, Just give me my thousand dollars, or say I choose nothing. Uh, those are not options you have to pick. Well, they might want to be options, but they're not part of I mean, why wouldn't you also pick the second box if you might additionally get

a million dollars? I don't know when. I feel like when you get into a thought experiments like this, they kind of beg for those kind of nitpicking answers, or at least I want to provide them. Um, Like any thought, when a thought experiment is presented, you can't help on some of them, but want to break it somehow, right, Well, of course, I mean that's something you should always play

around with. But given the constraints here, it seems like the obvious thing would be to say, okay, I want both boxes, because then I get the thousand dollars that's in box say no matter what, and then whether box B has a million or nothing, I either get another million or I just walk away with my thousand from box A. But here's the twist. The super intelligent machine has already guessed how you'll respond. If it thinks you're going to pick both boxes, then box B is certainly empty.

But if it thinks you will only pick box B then it makes sure or there's a million dollars in there waiting for you. But either way, the contents of the boxes are set prior to you making that decision. Now, this is really kind of change things maybe, I mean,

depending on what sort of decision theory you use. Right, if you trust the power of the machine to predict correctly, like you say that, no matter what happens, the computer predicts what I get your choices are one thousand dollars or one million dollars, then you should take the one million dollars by picking box B. But if you don't trust the computer to be correct in predicting what you're gonna do, then you should take both boxes because in

that case, if the computer was correct, you'll get at least a thousand dollars, and if it predicted wrong, you'll get the million and the thousand. So it's kind of a contest of free will versus the predictive powers of a godlike ai uh and how much you believe in either one, right, And it's an ability to predict your behavior or in your ability to have any free will

at all. So in you'd Kowski's timeless decision theory. He's says, the correct approach actually is to take box B, and then if you open it up and it's empty, you

don't don't don't beg for the other one. You just double down and still take box B. No, no, no, no. Back talks on this issue because you might Here's the here's the thing you might be in the computer simulation is it simulates the entire universe to see what you're going to do, and if it can trust you and your choice, then could affect the core reality outside of

this simulation, or at least other realities outside of the simulation. Yeah, the reasoning here is pretty wild, but it's operating on the idea that this super intelligent AI will be able to simulate the universe, that it will run simulations of the universe in order to predict what will happen in the real universe. And you could be one of those simulated agents rather than the real world version of yourself,

and you wouldn't know it. So, if you're in the simulation, you should pick box B because that will influence them Machene to predict in the real universe that you would pick box B, which means the real you will be able to pick box B and get one million or one thousand plus one million by taking both boxes. Unfortunately, the AI sucomputer supercomputer does not realize how indecisive I actually am, and I'm just going to simply ponder the

choice for the rest of my life. Well, I mean, this relies on the idea that that you would have looked into this issue or worked it out in order to decide which would be the optimal decision to make on the assumption of timeless decision theory. Uh, in many cases, probably people aren't going to be making the rational choices because a lot of times we just don't make rational choices. Now, if you're noticing that this type of decision theory relies

on a lot of assumptions, you are correct. It does rely on a lot of assumptions. But they are assumptions that are sometimes taken into account within people thinking about what a future technological super intelligence would look like. And it's the kind of thing that you know, you know, when I feel ideas like this in my head, you know,

and play around with the texture of them. It's hard to know where the line is between um being thoughtful and taking what's possible seriously, which I think is worth doing and and getting into an area like between that and getting into an area where you are starting to form ideas about the world based on extremely shaky assumptions, where basically you you begin to um reverse engineer health theology and other harmful ideas that we tend to associate

with religious worldviews and magical thinking. Well, we haven't gotten to the hell yet. Yes, the hell's coming. You need you need one more element to get there. Now. This next element, the basilist, comes in based on a background of thought in timeless decision theory, but also in another concept that Yutkowski has written about, known as coherent extrapolated

volition or CEV. And the short version of this, the simplified version, is that benevolent AI s should be designed to do what we would what would actually be in our best interests, and not just explicitly in what we tell them to do. So a simple example would be this, Let's say, um, I only use a variation on the

paper clip maximizer that Nick Bostrom has written about. You know, Nick Bostrom wrote about what if you program a benevolent AI you know it's not gonna it has no malice doesn't want to harm anybody, but you just tell it, well, I want you to collect as many paper clips as possible. And then what it does is it turns all the humans on Earth into paper clips. Uh, you know, it doesn't mean any harm. It's just doing what it was programmed to do. So there are dangers in kind of

naively programming goals into extremely powerful computers. Right, this could even happen if you were trying to program very benevolent goals into computers, you know, if you were trying to make a computer to save the world. What about So my version here is you tell a super intelligent AI that we want to eliminate all the infectious disease from the world. Think about how many lives we could save

by doing that. And in order to do this, it sterilizes the earth, destroying worldwide microbiomes, which cascades up the trophic chain or whatever. It kills everything on Earth. So if you have a super intelligence that you and you just directly program its goals and say here's what you

should do, you could run into problems like this. So the the idea behind the CEV thinking is instead we should just program the intelligent AI to predict what outcomes we would want if we were perfect in in our knowledge and and uh in anticipating what would make us the happiest, and then work towards those on its own, regardless of what we tell it to do, because obviously we can give it very stupid instructions, even if we mean, well, yeah, we tell it to love everybody, but there's a TYPEO

and we put dove everybody, and it turns everybody into delicious dark chocolate from tough. It's possible as well. This is how we get a Dove sponsorship on the podcast. But anyway, so, if you assume a super intelligence is using coherent extrapolated volition, that it's trying to determine what would be best for us, and working on its own terms towards those ends instead of relying on us to give it, you know, what are obviously going to be

imperfect instructions and commands, it might say predict. It might even correctly predict that the world would be a happier place overall if it did something bad to me. In particular, it might say, you know, from a utilitarian point of view, the world would be a much better place if it buried me, in a pit of bananas, so better for everybody else, not so good for me, as there's too

much potassium. But once you have that piece of logic in there, and combine that with the idea of of timeless decision theory, you can arrive at this very troubling thought experiment. The dark basil is ask yes, And the dark Basilisk of the Abyss has two boxes for us as well. One contains endless torment, and all you have to do to claim that box is nothing or dare to work against it. Uh. The other box is yours

if only you devote your life to its creation. And the prize inside that box well, not eternal punishment, which is a pretty awesome gift, if we're to choose between the two. Right, Yes, I would agree with that, though I would say not tormenting somebody that I don't know. Should you think of that as a gift. That's probably not a gift that that's the baseline, right, Yeah, Well, but you're staring down the dark Basilisk here, and okay, it's boxes are horrible. Well one is just less horrible

than the other. But the idea here is that just by knowing about the thought experiment, You've opened yourself up to that eternal punishment because now again your options are do nothing, work against it, or work for it, and only the third option will steer you clear of its Uh, it's you know, deadly done. Now here's where the really supposedly scary part of it comes in. You could think, well,

I'll deal with that problem when it arises. Right, So imagine there's some utilitarian supercomputer that's trying to even say it's trying to do good. Maybe it does. It doesn't have any malice. It just wants to save the world. But in order to save the world, it really needs me doing something different than what I want to do with my life. Well, I'll just make that decision when it comes up. What this thought experiment is proposing is that maybe you don't actually get to wait until it

comes up. Maybe this blackmail applies to you right now, retroactively into the past. So just by knowing about the thought experiment, you supposedly have opened yourself up to eternal punishment, or increase the probability of such. So imagine a simplified version. Say I am a computer, and I am the only thing in existence with the power to prevent global climate change from destroying human civilization. I can stop it, but people, they took a long time build me, and a lot

of damage was already done. So the idea is I might reason that it is good to blackmail existing people, or simulations of existing people, or even past people, in order to make them devote everything they can to building ME faster so I can save more lives in the long run. Of course, this incentive would have to apply

to the past once I exist. I already exist, right, So the only way the past people would have an incentive to respond to this blackmail is if they predicted that this blackmail might occur and took the idea seriously and behaved accordingly. Right. So, thus the idea, the idea itself puts you at increased risk of being on the real or simulated receiving end of this a causal, retroactive blackmail if you know about it. And this is why this idea would be classed by some as a potential

information hazard. And I'll talk more about the idea of an information hazard in just a minute. But one of the things I think a lot of people writing about this topic miss out on is they, for some reason get the idea that Roko's post that this thought experiment is is generally accepted as correct and plausible by Yudkowski and by the Less Wrong community, and generally by the people who put some stock in whatever these ideas are,

timeless decision theory, coherent, extrapolated volition and all that. It is not widely accepted among those people. It was definitely not accepted by Yukowski. It was not and is not right. It is not the dark, deep secret of of less Wrong. But unfortunately, after the post came out, it was heavily criticized,

and then it was banned. And I think a lot of people looking back on the idea have said, oh, that was not such a great thing to do, banning the idea, because it gave it this allure of like it was almost as if by banning it that made it look like the authorities had concluded that this idea was in fact legitimate and knowing about it would definitely harm people, and that is not the case, right. And it also, I mean it added to the forbidden fruit

appeal of it too. Write I mean it's like, oh, I'm not supposed to know about this the pony up I want to know, and now people are talking about

it all over pop culture. I mean, I have actually resisted the idea of doing a podcast on this before, mainly because not because I think it's seriously dangerous, but because I think, well, is there any benefit in talking about something that I think is very unlikely to have any real risks but in some extremely unlikely chance, or what appears to me to be an extremely unlikely off chance, could actually be hurting people by knowing about it, you

know what I mean, It's like, what what is the upside? But at this point enough people who are listening to this podcast probably already heard about it. They're probably gonna hear about it again, and that, you know, sometime in the next few years, through pop culture whatever. It is probably better to try to talk about it in a responsible way and discuss some reasons that you shouldn't let

this parasitize your mind and make you terrified. Right. One of the reasons we're talking about during October is because it is a suitably spooky idea. It is a troubling thought experiment, and we're leaning into some of the horror elements of it. But I also do really like making sure that we explain the mythic and folkloric origins of the basilist itself, because the basilisk itself is this wonderful mix of just absolute horror and desolation and just also

just utter ridiculous nous. I mean, it's it seems like one of the main ways that you defeat the mythic basilisk is through in a way, through humor, running around with a chicken and a weasel and a crystal globe and realizing that it is truly a little king. So I think it is it's worth remembering the little king and talking about the great Basilisk. Well said, I think that's a very good point. But anyway, I did just

want to go ahead and hit that coy. Yet that a lot of people, for some reason seemed to use this idea as like a criticism I'm not like a less wrong person, but as a criticism of the less wrong community, as if this idea is indicative of what they generally believe, and it's not. It was a heavily criticized idea within that community. Right. It's like thinking that

wherels of London is the warren Zevon song. Really, you know, you have had a rich discography with with many much better tracks in my opinion, it's just that's the one that got the radio play. Now, Robert, what was that you You said that you saw something about this idea

and a TV show. Now they're talking about it on TV. Yeah, so this is this is kind of fun because I think a listener had had brought up Rocco's Best List as a possible um topic, and you said, oh, I don't know if we want to want people knowing about it,

and I well, well, I mean, but yeah, caveats. Okay, not because I think it's legitimately dangerous, but because what is the level of tolerance you have for talking about ideas that are not necessary to talk about and that represent a class of something that people could think was dangerous to know about. It might cause them terrors and

nightmares and stuff. Right. So so my response to that was, well, I'm not going to look it up, not because I was afraid of it, because I'm thinking, well, that could make for a good podcast if, like Joe is telling me about it for the first time, whatever this idea is. But then I was watching HBO Silicon Valley and they explained it on Silicon Valley and I and I realized, well, the cats out of the bag there. But yeah, there's

a character named Bertram Gilfoil who's a fun character. He's like a Satanist programmer live in Satanism of course, and uh. And he gets rather bent out of shape over the concept as it relates to the fictional Pie Piper Company's involvement with AI. And he starts like making sure that he's created like essentially a paper trail in emails of his support for the AI program so that he won't

be punished in the digital afterlife. Well, hey, this comes in again when we remember when we talked about the machine God in the Machine Odd episode where that I've forgotten his name now, but the Silicon Valley guy who's creating a religion to worship artificial intelligence as god. And I,

you know, I don't really love that. One of the things that comes out when he explains his mindset is that he seems to be kind of trying to, in a subtle way, be like, look, you really don't want to be on the wrong side of this question, if you know what I mean. You know, you want to be on record saying like yes, I for one welcome our new machine overlords. I'm I'm expecting he'll buy a lot of our all Health of Great Basilicity shirts at our store, available by put clicking the tap at the

top of our homepage. Stuff to blow your mind dot com. Oh man, you are plugging like hell. But anyway, I'd say it's unfortunate the way this, like single Internet post and then all this fallout related to it played out because it lent credence to this scary idea, even though the basketball scenario I think is implausible, and and the people of that community seem to think it was implausible.

The idea may constitute sort of part of a class of what's known as information hazards, defined by the Oxford philosopher Nick Bostrom, who we mentioned a minute ago. Uh Bostrom has written a lot about superintelligence and information hazards would be quote, risks that arise from the dissemination or the potential dissemination of true information that may cause harm or enable some agent to cause harm. So this is not talking about the risks of say, lies or something

like that. This would be the idea that there's a statement you could make that is true or plausible that by spreading actually hurts the people who learn about it. And this is exactly the reason as you're mentioning that's referred to as a basilisk. It can kill or in this case, increase the likelihood that something bad will happen to you if you simply look at it or know about it. And so, even though the idea is implausible, the dissemination of this terrible idea would seem if certain

conditions are met to increase its plausibility. Right, you're increasing the incentive for this future AI to blackmail versions of you in the past, just simply by acknowledging the incentives could exist. Anyway, Maybe we can get out of this section for now. But I was just trying to work out, like, why have I been hesitant to talk about this on the show even though people have been requesting it. But I don't know if it's on TV shows, it's all

over the internet. It's fine. Now the basilisk is out of the bag. All right, Well, we're gonna take a quick break and we come back. We'll continue our discussion and we're gonna discuss something that that a number of you are probably reminded of. As we've been discussing this, we're going to talk about Pascal's wager. Thank you, thank you. Alright,

we're back now, Robert. One of the things that this idea of Rocco's basilisk flows from is thinking about decision theory, right, how do you make the best decision when you're presented with certain options? And there there's they're a little payoff matrices people fill out where they say, okay, given these options, what actually would be statistically the best decision to make. But this is not the first time people have applied

these kind of decision theory matrices. Two ideas about your eternal soul or your eternal well being, or the idea that you could be tortured for eternity. Yeah, we can

go all the way back to Pascal's wager. For instance, technically one of three wagers proposed by French philosopher uh Blas Pascal is that the correct French that might be I think I would just usually say Blaze, Blaze or or Blassie one of the three old Blassie Pascal Blaze Pascal who lives through sixteen sixty two, and he argued that everyone is essentially betting h on the existence of God. The argument for theism is that if God does exist,

then well there's an advantage in believing. But if God does not exist then it doesn't matter. But since we can use logic to tell if God exists or not, there's no objective proof. We can only make our choice given the relevant outcomes. It's looking at your religious beliefs and saying, oh, you're nonbeliever. Huh, hey, what have you got to lose? Exactly? Yeah, Pascal wrote, let us weigh the gain and the loss in wagering that God is. If you gain, you gain all. If you lose, you

lose nothing. Wager then without hesitation that He is. Now I've got some things I want to say about this, but you had some stuff first. I think, well, yeah, there are there are a lot of issues that one can take with this based on knowledge of world religions, philosophy,

statistical analysis, etcetera. And and yeah, I have to admit that it can start to break your brain though a little bit, if you think too hard about it, Like I found in researching this this podcast, really thinking about how I would react to Pascal's wager if I was like forced to make an answer to to to to FORMU late an answer like that, Like you mean if you were given good reason to think that there would be punishments for not believing in God or something, right,

And but I didn't know which religion was correct, and I had to like proceed based upon the relevant level of punishment for unbelievers in various religions, and like which one is most correct for like you just think that would mean it would what would be rational to choose the religion that has the most lurid hell? I guess, But then that really feels like losing, doesn't it? Um? You know it? It's certainly though, reminds me of the more boiled down versions of this that you encounter in

various forms of Christianity. Right, except Christ go to Heaven. Reject Christ go to Hell? But what about the people who haven't been given the choice yet? Right? That's ah, that's the other concern. Well, if they're all default hell bound, then God comes off as it comes off as a bit bad, right, Like what kind of God is that?

But if they haven't out, if they're spared hell fire or at least you know their section for Dante's limbo of virtuous pagans, then is the missionary doing them a disservice by even presenting them with the choice like why do you even ask me? Because now I have to I have to, I have to devote myself or not, like now I actually have You know, I was just going to go into the uh, you know, default limbo category or the default heaven before and now now I'm

actually at risk of hell. Well, that means certain theories of damnation mean that presenting the Gospel to someone as an information hazard, you potentially harm them immensely by telling it to them. And I think part of this just to to to go beyond like the the actual um U wager here is I think a part of the issue here is that we're using evolved cognitive abilities that are that are geared for smaller, though often important choices, and here we're trying to use our imaginative brains to

create a conundrum that can outstrip those abilities. Yeah. Well, I mean that is what we do in philosophy, right, We're constantly using our brains in situations it was not really made for um and just trying to do the best we can. But I mean it's quite clear that motivated reasoning is often a thing when when we're trying to be rational was just failing. But of course this is how we train our brains for rational thinking, often

oftentimes exploring these various outsized ideas. You know, there's so many ways. I think Pascal's wager kind of breaks down because it's obviously there's the thing you pointed out about there's more than one religion, right, you know, it's not just like do I believe or not? It's like which one? But it also it implies again this is like a theological question, but it would seem to imply that God can be tricked into thinking that you believe in him

if you simply pretend to. I guess Pascal had I think maybe a more sophisticated way of looking at this, you know, that like live as if God exists or something. But it but the wager is often used in very unsophisticated ways. Yeah, but it implies that it doesn't matter to him what you actually believe, only what you outwardly claim to believe. Though then a again, the funny thing here is this might be the case with Rocco's basilisk, Right, what would this machine? God care what was in your heart?

It only cares whether you help it or not, or whether you you know, proclaim fealty to it or not. Yeah, that's why the T shirt is so important, Joe, because if it, if it knows you purchase that T shirt, then you're you're square, You're covered. Okay. Yeah, As as best Singular pointed out in that Ian Magazine piece of reference earlier, she says, quote the secular basilist stands in for God as we struggle with the same questions again

and again. So her argument is that we kind of reverse engineer the same problem again through our contemplations of of super intelligent AI. Yeah, I guess the You get into a plausibility question here, right, you get into a question about is uh it actually possible to make an artificial intelligence that is functionally equivalent to God. I mean, we're not thinking we could build an AI that would break the laws of physics. So it might be able

to run simulations of the universe. They have cont you, conscious agents within them maybe for all we know, and they could break the laws of physics inside them. But yeah, I mean, could that even happen? And the issue is we don't know. We don't know whether that could happen or not, So should we behave as if that is a plausible thing to be worried about and to consider, or should we behave as if that's just not really something you need to concern yourself with. I don't know

how likely or unlikely it is. And if your your fears are related just to the idea that you're you

could be digitally resurrected, uh for torment and the basilisks dungeons. Um, I mean that that of course would depend on to what how much docuputing the idea of digital consciousness and the whole philosophical question we've we've touched on here before is that you I mean, it's just a copy of me, right, So why I mean, I ultimately can't do anything about you know, a thousand different basilisks creating a thousand different

copies of me and tormenting all of them. Um, there's still to ourge extent, it's just just rowing me in effigy. There are actually a bunch of reasons I wrote down to doubt the plausibility of the basilisk. We could do that now, or we could come back to that later. I don't know what you think. Yes, let's to do, but I will add the idea of being tormented digitally. Uh, this does become more dangerous. I guess if you believe you might be in a simulation right now exactly, then

then things are a little more dire. But that's again that you might be that you might be, Yeah, but I believe there's plenty of reason to believe that you are not. Okay. So if we're talking about how to defeat the basilisk, how to get out of this, uh, this this prison of the mind. If you're feeling a little bit um um bleak of heart right now because of this idea, then Joe's got the remedy. Well, I'm not.

These are not all the reasons you should doubt the basilisk, but this is some of them that I could think of. Number one depends on the creation of super intelligence, which I think is not guaranteed. Some people seem incredibly fatalistic about this is just absolutely inevitable. We will have super intelligent god like a I that can do anything, and I think that that is just not guaranteed at all.

I'm not ruling it out, but I think, for example, there are some theories of intelligence that say the prediction of super intelligence actually is maybe not taking Seriously, what intelligence is that you know that they are actually different kinds of intelligence that are useful in different ways, and machines can't mimic them all functionally, or can't mimic them all correctly, all at the same time. I don't know if that's correct, but that's at least one. That's one

hurdle it has to clear. Could get knocked down there, but okay, maybe we could create a super intelligence. Even then. Multiple aspects of the Rocco's basilist scenario depend on the reality of some version of mind uploading, or the idea that your brain and in addition, your conscious experience could

be simulated perfectly on a computer. And one reason it depends on this is that timeless decision theory operates on the assumption that the real you and the simulated copies that the computer uses to predict your behavior would be the same and would make the same decisions as the

real you. Another reason is related to the punishment. Now, one way of course you could imagine the great basilisk thing is that if the machine comes to power in my lifetime, it could just punish the real, physical, older version of me. In reality is the payoff of this a causal blackmail. But the other way you could imagine it, in the way that it is much more often portrayed in the media, is that it makes digital copies of

my consciousness and punishes them in a simulated hell. And that, of course, would also depend on the reality of some version of mind uploading, or of the ability of a computer to simulate a mind and for that simulated mind to actually be conscious. As I've said before, I'm suspicious of the idea of conscious digital simulations. I'm not saying I can rule it out, but I also don't think

it's a sure thing. Any scenario that relies on the existence of conscious digital simulations needs a big asterisk next to it that says, if this is actually possible. Yeah again, is that me? Is that just me and effigy? Is that thing actually conscious that you're tormenting? I mean, granted, it still sucks if there's a super intelligence creating digital people and tormenting and it's a dark, rancid dungeons in the future, but um, it's not necessarily quite the same

as torturing me. Right well, if you just care about yourself. It also depends on the possibility that you could be one of these simulations. It's possible that you could not be one of those simulations. There's something that would rule it out. Maybe their type of conscious Maybe they could be conscious, but that consciousness is fundamentally different from yours

such that you could not be one of them. Another big one, and this is a big one that uh, you know, like we said earlier, I think sometimes Yudkowski gets unfairly associated with the basilist as if he has advocated the idea, and he has not. He has said, you know this, this idea is trash, and uh, there there are many reasons to doubt it. But even though he has said, like even though I doubt it, I

don't want it disseminated. Um, but he says, you know, a good reason to doubt it is there's no reason to conclude it's necessary for the basilisk to actually follow through on the threat. We're saying that it's going to be relying on us to come up with the idea that it in the future might blackmail us if we

don't help it now. In order to get us to help it now, right, we should be working and donating all our money and time and resources to building it as fast as possible, because we came up with the idea that it might torture us if we don't. Even if you accept that Yudkowski has he's pointed out that there's no reason once it's built, it would have to

follow through on the threat he's written. Quote. The most blatant obstacle to Rocco's basilisk is intuitively that there's no incentive for a future agent to follow through with a threat in the future, because by doing so, it just expends resources at no gain to itself. We can formalize that using classical causal decision theory, which is the academic

standard decision theory. Following through on a blackmail threat in the future after the past has already taken place, cannot, from the blackmailing agent's perspective, be the physical cause of improved outcomes in the past, because the future cannot be the cause of the past. Hey, basilist, why are you

tormenting a third of the population for all eternity? Oh, I said, I would, Well, yeah, I mean exactly no, if it it didn't say it would, right, It just had to rely on the fact that in the past people would have come to the conclusion that it might.

You know, you thought that I would. I didn't want to disappoint, But actually, if a basilist could be created, it seems like the best case scenario for it would be everyone subscribes to this idea and works as hard as they can to build it, and then it never follows through on any of the threats. Right, the best case scenario would be people act as if there is a threat and then there is in fact no follow

through on the threat. It's really a win win for the bathlist, Yes, and then he can maybe you can even shed that name Basilist. They're like, we don't even have to call it the Great Basilist anymore. We can just call it, uh, you know, Omega or whatever it's its name is now. I want to be fair that a lot of what these people do is like the less Wrong community and all that they deal with, Like, should there be alternative decision theories that guide the behavior

of superintelligent aies. Maybe it doesn't use classical decision theory, Maybe it uses some kind of other decision theory, and because on some other decision theory, maybe it could decide to actually follow through on the blackmail threat. I think that is where some of this fear comes through that like, oh, maybe by talking about it, we are actually causing danger because maybe some other decision theory holds. But Yudkowski does not think that's the case. Also, one more thing, it

depends on the basilist. So if you think this scenario could be real, it depends on it not having ethical or behavioral controls that would prevent it from engaging in torture. Yeah. And I think if thinkers like the you know, the Miria people, the Machine Intelligence Research Institute people, succeed in established what they're trying to do is establish a philosophical framework to make AI friendly, to make it so that

it is not evil and does not harm us. And if they successfully do that, then this shouldn't be a problem, right Because Yudkowski has argued that a being that tries to do what's best for us would not engage in torture and blackmail, even if it's doing so in service of some higher good, because doing torture and blackmail are

actually not compatible with human values. I agree with that absolutely, and I actually would go as far to say I think that's something people should keep in mind when they're uh, when they're they're choosing their religions as well. Yeah, I can certainly see how you can make that argue. Yeah, it's like, what do I love about my faith? Is that the blackmail and the torture? Or is there something that brings something else to the table that is worth

living forward that makes life better for everybody? Like, like I feel like, is what should be important about one's faith? Now, I think some people might be saying, like, wait a minute, though, if you're just using utilitarian ethics, right, wouldn't wouldn't any methods be good if the if the ends justify the means, right? That that's I think a naive understanding of how people

think about utilitarian ethics. If you want to bring about the greatest good for the greatest number of people, couldn't you do that by being really cruel and unfair to some smaller group of people? And I think generally there are versions of utilitarianism that say, well, actually, the answer there is no, you couldn't do that, because even though you might be bringing about some better material circumstance, it is actually corrosive to a society for things like that

to happen, even if they don't happen to many people. Right, you say, what if I could make everybody on earth ten percent happier on average by say, burying somebody in a in a pit of bananas once a year, so that you buried to death with bananas. Even the people who are being made happier could very easily look at that and say that's not fair and it makes the world worse and I don't want it. And thus that

actually would be a subjectively relevant state. So we've talked about AI risk on the show before, and you know, one thing I feel like I still have not been able to make up my mind about, despite reading a lot on the subject, is that I don't know whether it makes sense to take um to be super worried about AI super intelligence and the risks associated. I mean, I do think it's worth taking seriously and thinking about.

And I think people who want to devote their attention to how, you know, how dealing with the control problem and how you would get an AI to do things that were good for us and not harmful to us, that that's fine work. And I don't ridicule the people who work on that problem the way some people do.

But on the other hand, I worry if by focusing exclusively on sort of the machine god the super intelligence, were sort of ignoring um much more plausible and current threats that the ways that AI is already very plausibly in a position to hurt us today or in the very near future, and not depending on any outlandish assumptions. The way it's already and will soon be used as

a cyber war weapon. The way it's hijacking our attention and manipulating our opinions and behavior through social media and devices. This is some of what our Scott Baker talked about with his fears about AI. You don't actually need super powerful AI to do a lot of damage. It just needs to manipulate us in just the right kind of ways. So not the great basilisks so much as all the little basilisks that are out there, the little grass snakes, grass with the tiny crowns. They can do a lot

of damage. And again I just want to be clear, I'm not saying we should forget about super intelligence. People who are working on that. If you find that interesting that I think that's fine, Yeah, work on that problem. But there but I think it's a longer shot, and there's a lot of current and near future AI threat that is really worth taking very seriously. I wish people more people were devoting their lives to say AI cyber

weapons that are in development right now. One last issue I think we should discuss before we wrap up here is Okay, so we don't think this potential information hazard is actually an information hazard, Like we don't think it's actually potentially that dangerous. But Yudkowski has made the point that even though he doesn't think the basilisk is plausible, the ethical thing to do with potential information hazards is to not discuss them at all, since it's possible that

they may be. Maybe maybe you're misinterpreting the ways in which they're implausible. Maybe this idea is actually valid, is actually relevant, and by spreading it you've harmed a lot of people. But I also think that this could mean that it's it's possible that despite the basilisk not being plausible, something good has come out of the basilist conversation because it encourages people to think of the idea of information hazards. Maybe Rocco isn't true, but there could be other ideas

that are both true and potentially harmful to people. Just by entering their minds. And the lesson from this is we should prepare ourselves for those kinds of ideas. And if you have discovered one of those ideas and there is literally no upside to other people knowing about it,

keep it to yourself and don't post it on the internet. Well, I feel like I do encounter thought hazards like this from time to time that they're often presented in pamphlets or little booklets, uh, generally with you know, a clever illustration about the coming into the world. Actually brought some of these into the office recently. I found them at a park in rural Georgia, and uh, and I think I told you it's like, Uh, you have have a

look at these. You may find them interesting, but UH, do destroy them when you're done, because you know I did. I did in the wrong hands. These thoughts can be dangerous if they have some sort of a like a harmful view of society that that people may buy into. Well, I think you were comfortable sharing uh, malicious religious literature with me because you do not think there's a possibility that that literature is true and would harm me if I knew it was true, like you think it is false,

So to you, it's actually not an information hazard. It's just like an idea hazard. Uh. The real crazy thing would be if you came across a pamphlet and you read it and it's the equivalent of this raving malicious religious literature, except you were convinced it was correct, if it was more like that ring video I brought you exactly. That is one of the things I've often seen on

the internet, this idea compared to the ring. But you know, on the other hand, I do have to I am reminded, you know that like the idea that that any kind of knowledge is forbidden or is secret, like that doesn't really jive well with just the the general mission of science, of course not yeah, but I mean that would be part of the problem of like we're not prepared for information hazards, right, because in the past it's been the case that almost anything that's true is good to spread, right,

unless you're spreading lies, information is good to share. It's just possible we should acknowledge that maybe there is such a thing as a fact that or a fact or an idea or a theory or something that is true and correct. But it would hurt people to know about it. I can't think of an example of anything like that, but if there is something like that, we we should be ready to not spread it when it occurs to us.

All right, fair enough, Well, I want to close out here with just a one more bit of basilisk wisdom or anti basilist wisdom. And this comes from the poetry of Spanish author Francisco Gomez de Gravado e Viegas, who Live five. This is translated and it's referenced in Carol Rose's Giants, Monsters and Dragons quote. If the person who saw you was still living, then your whole story is lies, since if he didn't die, he has no knowledge of you, and if he died, he couldn't confirm it. So I

was thinking about that with the stories of the basilist. Yeah, I was like, wait a minute, that How would you know if you could die just by looking at something? How do we have this description in the book? Yeah, it's uh, there is a there's an authorship problem with this. Yeah, whose whose account is the basilist? But at any rate, I think it's a nice like final you know, sucker punch to the basilists in general, but also a little

bit to the idea of the great basilisk, right. I hope you were not leaving this episode with with terrors about future digital torment. I think that is not something that you should worry about. Indeed, I'm not worried about it, and instead of worrying about it yourself, you should head on over to stuff to Blow your Mind dot com. That's the mother ship where you'll find all the podcast

episodes links out to our various social media accounts. Who find the tab for our store, you can look up that bassilist shirt design we're talking to out and uh and that's a great way to support the show. And if you don't want to support the show with money, you can do so by uh simply rating and reviewing us wherever you have the power to do so. Big thanks as always to our wonderful audio producers Alex Williams

and try Harrison. If you would like to get in touch with us to let us know feedback on this episode or any other, to suggest a topic for the future, or just to say hi, let us know whether you carry a weasel around In case of a basilisk encounter, you can email us at Blow the Mind at How Stuff Works dot com. Oh, I'm hearing that transmission again. That weird snow. What is that? Oh Hell, the Great Bassilisk. All Hell, the Great Basilisk. Oh Hell the Great Bassilist.

All Hell, the Great Bassilist. Hell. There the great basilists call Hell the Great Basilisk, Hell the Great Bassilisk. O Hell, the Great Basilisk,