TechStuff Turns On the Dark Web

Strickland Hi high above the North Pole. On the first day of nineteen sixty nine, two professors of English literature approached each other at a combined velocity of one thousand, two hundred miles per hour. Hmmm did they collide? You'll have to read the book to find out. Done. Done. So we are today talking about an interesting subject. Definitely something that is beyond the eyes of mirror mortals, unless, of course, you will put the right software on your

computer or you know what you're doing. Yeah, more or less, Okay, beyond the eyes of people like me, because I'm such a I'm such a service level kind of guy. We're talking about the dark web or the dark Internet, or the deep web, or the undernet or the deep net, or the invisible web or the hidden web. There are a lot of different names for this. Yeah, so there are two main definitions we need to get out of

the way. Yes, because there are people think of different things when these terms come up, and both both things are real and need to be addressed. So here's the less interesting one, at least in my opinion. There are machines that either were once connected to the Internet or were once connected to a network that subsequently was connected

to the Internet, that are no longer reachable on the Internet. Okay, so let's say, like their entire military networks that were part of arpanet back when arponette was a thing, and Arpanet, if you guys don't know, was a predecessor to the Internet. It was where a lot of the protocols that were developed for the Internet first were used. If only somebody wrote an article about that for how Stuff Works dot com. Hey, and funny, you should mention that I did write an

article about that how arpa Networks. But yeah, urpennet actually a very interesting story. Yeah, it's and that is the story of creating a network that can let different machines uh communicate with one another. You know, and before networks were mostly composed of machines that were essentially identical to each other and they were all generally within the same space, like same building. But Arpanet was an attempt to create a network that would allow different machines running on different

operating systems and different locations communicate with one another. And it grew substantially over the years that it existed, and it started to incorporate other types of networks, including military networks and research organizations. Well, some of those military networks did not keep pace with the changes that were aid when the Internet began to emerge. Now, our bonnette and

the Internet are not the same thing, not technically. The Internet is a is a network of networks, of which our ponnet was sort of kind of one, But our ponett's essentially been offline. That's another example. Is that stuff that was on our ponnet that's no longer accessible by the Internet. Um, sometimes these machines themselves have failed and they are no longer reachable, so that information is the at least the original source of the information is lost.

You might be able to find bits and pieces of information on other sites because people have have cited it or stolen it or whatever, but the actual original source of that information is gone. You know what occurs to me, what's that that we should point out to that that that information may not have ever been on the web. Um because as as a lot of our listeners know, but maybe not all of them, Uh, the the Worldwide Web is a part of the Internet. It is not

the entire ternet. There's information that is available on a computer network that can be accessed through different means, and so calling it the deep web in that case, in the in the definition that you were just talking about, the sense that that you were just talking about may not necessarily be completely accurate because it might be found

in a different form. Um. Not that I'm saying that you're wrong, more of a clarification so that that people understand it does a great clarification, because, yeah, the Internet comprises so many different ways of sharing information email for example, one way, file transfer protocol, another way using net us net HTTP being the web based way of yea of

sharing information. And yeah, when we talk about deep webs, some of this information is not necessarily the kind that you would access with a normal web browser, right exactly. So that's a good point. You might have to have some sort of emulator, or you may have to have a totally different kind of program in order to access that information, and that sort of leads into the second sense.

I think, yes, yes, So, so that first sense is stuff that is no longer accessible, either due to machine failures or the fact that just did not keep pace with the protocols, or it was disconnected for a reason there. Yeah, just though it is now its own intranet, but is no longer connected to the Internet at any rate. That's stuff that maybe at one time was connected to the Internet, but now no longer is, and therefore you cannot get to it. That's why we call it dark. That's that

one definition. Now here's the more super cool, awesome, spy scary definition. Well, and in its purest sense, I would say it's not spy or scary at all. Um. You think about the depth and breadth of the Internet as you see it on a search engine. You know, you put in a search term, you get three hundred thousand hits plus for for a topic, and you go, wow, there's a whole lot of information out there. Well, that's only the information that the web browser that the search

engine knows to go to find. We call that the surface web. Yeah, yeah, and it's overused as as a metaphor. But the whole Iceberg thing really applies here. It is really a very very small section of what's available on the internet as a whole. Part of it's because that that's what's available on the web, and there are other protocols.

It's not that Google, for example, is not searching. You know, it's not necessarily collecting all the information that's on on US net sites and things like that, so you may or may not find the information that's out there. Uh some people don't want their site indexed, and they can choose not to just by putting in a little bit of code at the top of their their their page. It will It's essentially an instruction saying, hey, web crawlers,

skip the site. Yes, do not index this site that I do not want an index and the web crawlers won't, they'll ignore it, they'll move on. And so then you have a website that is not indexed by any search engine and the only way to get there or as if there is another incoming link from somewhere else. Uh So, so someone's build a website and they've gone outgoing link to this other website that doesn't have any that's not indexed by websites or web search engines rather or you

physically give out the U r L two people. Well, and then then there's stuff that's dynamically generated, such as the search engines results page such as, um, let's say you go to your bank and you want to see, uh, you know, the your balance on your account. You log in and you you know, check the balance on your account. Well, that page that you generate by logging in checking at you know, to fourteen pm and twenty six seconds, that page is not indexed by anything. So that is a

dark web, if you will. That's a dark web page that you alone are accessing and no one else really has access. So so dark web includes all these databases that have have terabytes and terabytes and terabytes of data that again will only appear in very specific sets of circumstances and otherwise doesn't. So therefore they are not indexed. Yauto bites even mostly just because I wanted to say, y auto bites, Yes, there is such a thing. I'd

like to grab some yauta bytes after this podcast. Um so yeah, I mean, and and it's purest sense, the dark web, the invisible web, the invisible net. These are things that that just aren't. They're hard to find. They're not something that you're going to find simply by uh, you know, surfing around or typing in a couple of terms in the search engine. It's not gonna come up. Yeah.

So there there's a whole lot out there. I mean, um, you know, I I use part of my website as a sandbox just to test different technologies and play around with it so I can learn more about it. And I, you know, I put security access on it because I did. I'm just fooling around, you know, I'm gonna type bladdy bladdy bladdy blah. Who look at that. I could change the fonts, I could, you know, And I want to

see what this under construction gift looks like on my site. Well, you know, this looping middi is so much more awesome than that looping middi. Wow, this browser totally still supports that marquee tag. This is where Jonathan is clear that Jonathan has not built a website since anyhow. Yes, that is part of that all as part of the dark Web, and and therefore there is nothing particularly sneaky. You have

very funny. Look. I was just I was just thinking back to my oldest stinking back to actual websites I've built back in. That's the problem. Alright, sorry so but but um, you know, we did have somebody who was asking about this, and we have not revealed your identity dark web person. You're is safe with us deep web anyway or her. Let's not narrow it down too much him or her. So the thing is there is and you think about it. Okay, I get it, but that's

not really all that interesting. It's not necessarily but people are using the the inability to find this stuff, uh to their advantage. They might be the hiding secret information or you know, information that might get them in trouble with the law, or they just don't like the idea of companies, websites, governments being able to track all their movements. What. Yeah, there are plenty of people this century, right Have they talked to Mark Zuckerberg? Have they have privacy is a

thing of the path? They read George? Orwell, uh, there it was, and look how far we've come the uh. Yeah. The one of the concerns about using the Internet for a lot of people is that a lot of the stuff you do is trackable. There are records of what

you do. I mean, this has been in the news recently because Google has changed this privacy policy where it's going to be sharing the information that gathers across multiple Google products in order to shape results for you personally, which means that it's actually actively using your history of navigating the web. Uh, and that freaks a lot of

people out. Yeah. Yeah, well, speaking of people want to know the as of the day, we're recording this that happened yesterday, March one, and uh, you know, people have been making a big deal out of it ever since Google had announced it, and plus plus they put it on all those sites. There's a little banner at the top seeing, uh, this stuff matters, which I thought was

kind of funny. And they actually make it easy for you to look at what the history of your web searching is, so that way you can choose to and you can get rid of it. Yeah, you can choose to delete it if you want, you can keep it. I mean, the purpose of that, from Google's perspective is

to give you the best service possible. But there are people who have brought up the concerns and I think they're completely legitimate concerns that the fact that you have this history that's personalized and visualized to a very specific user. That's troublesome because what happens if another entity comes in that isn't concerned about giving you a great experience, but it's more concerned about finding out exactly what you've been

up to for you know, a nefarious purpose. And uh, and that might be that you could be a perfectly innocent person and still have problems arise because there there's this information about your habits when you're out there serving the web or using the Internet in some way. And so there are people who have used this, this part of the web, this kind of idea of the uh. This this anonymous, untrackable sort of web presence to be

able to navigate through without leaving a trail. And in some cases it is to do something that is not necessarily ethically uh in the green right. I mean it's some stuff. Some of the stuff is really illegal, unethical, nasty stuff. Yeah, well it's it's like everything else. You've got the technology to do this. It can be used for good. Pause, Yeah, if we're for good or bad.

Like for example, Google basically says when you sign up, hey, look, Jonathan, just between you and us we're gonna collect this information, and as a result of us collecting this information, we're gonna be able to tailor this for you. We're not gonna look at your records. We don't care where you go. This is all gonna be stored on a computer in between you and us. You'll get a really great experience. And Jonathan may say, well, okay, I mean, hey, just

between me and you guys, I think we're good. You know, I don't. I don't mind that you're you're collecting this stuff because I want a positive experience. I want to visit a site in my password, just to go in there,

and I'm blam, I'm right there. The thing is, what if somebody breaks into the server, or what if they're there, there's an unethical employee or or somebody who gets ticked off at the company and says, you know what, I'm going to steal some customers information and make this company's life miserable by giving him bad pr I mean, it could be used for something like that, although the company

may not necessarily be trying to do that. On the flip side of that, uh, people might be saying, you know what, I'm just uncomfortable with you having that information, and the other people might be saying, I'm uncomfortable for you having this information because I don't want to get thrown in jail. So you know, there's a there's a positive and a negative for all these things. Right, So you've got you know, you've got people who are using

it to do stuff like pirate material. Yeah. So if you created a database online and you did not have any pointers to this site, so no web search engines are going to be pulling up this site, and you distribute the information some other way to the people that you want to visit your site, then theoretically you could host stuff and the the likelihood that you would get

caught would be lower. Right, So there are people who do this where you know, you hear about like the big peer to peer network X and everything, because those are the ones that get a lot of focus and they are searchable. But then there are other ones that are not searchable, and they there are plenty of them that exist out there. It's still just as illegal to use them, but they are they are evading detection more effectively.

I think. You know, the organizations that are concerned about intellectual property have to make sure that they get the services of people who are aware of the dark web

in order to be able to police that kind of stuff. Um. But but there it's also used for things like a journalist who might be overseas in a country that is not so concerned about offering the press free speech as other countries, and that journalists may want to be able to publish reports that are unfavorable to a regime in that country, and using something like dark web in order to get that information out could protect the journalists or

the journalist sources. So let's say like the journalist is overseas talking to a government official UH and trying to communicate with this government official who does not approve of the regime's policies. Well, that official could very potentially be at in serious danger for sharing the information. Using some sort of untrackable anonymous UH software or doing this over the web in a way that's not not searchable or

trackable would be very important in that case. And so there are tools that are out there for people to use. And again, just like Chris said, they're the tools are are neutral. The tools are tools. Yeah, they're just tools. But there are tools out there like me so like free net that that's an example. Now we've talked about

free get before. UM it was some time ago, I think about a year ago as a matter of fact, because we were talking about the UH using the internet to communicate UM protests and other information in countries where UH the government is aggressive about finding and punishing people for UM speaking their mind in that country. UM. Of course, we've over the last couple of years, we've had a lot of that in many countries around the world, where people are UM getting the word out about stuff that's

going on, people who are being jailed for being dissidents. UM. People are trying to find ways to communicate with one another, to establish protests. UM. The government is is watching the law the the law enforcement authorities in that area are are watching to see if they can crack down on these protests to keep it from becoming worldwide, to keep

it from gathering more support. UM. So they these folks are trying to find ways to share information without bringing down the authorities on them, and software like free net is is one way that they can do that. And in fact, some people were setting up UH servers over the past couple of years where people could speak their mind and it wouldn't disclose their location, um, which and a lot of people who were supporters of democracy for

these these uh governments that were not necessarily so democratic. Um, you know, they were saying, oh, well, this is a good thing. We're we're giving people an opportunity to speak their mind, that we're giving them freedom of speech where they had done before. Yeah. Another example is uh the Onion router or tour. I've used that myself on my

own computer. As a matter of fact, you may you may have heard of tour and tour is a part of a project called Anonymity Online, and it's all about being able to to access the Internet anonymously where there's

not going to be any tracking. Um. You have to pair this with some other software sometimes depending on what it is you're trying to do, but in general, what it's trying what what it's its purposes is so that you can go through and find out what you need to find without leaving a trace of what you're doing, which can be really important depending on what is like maybe you're a whistleblower, Maybe you're in a huge company and you've seen something that's truly unethical and you feel

it is your responsibility since you know about it to reveal this, but you also know that if you do this you could have some serious problems in your personal life further down the line. Well, that's sort of the kind of tool you might want to look into if you were to do such a thing. Right. So, the way Tour does it is kind of interesting. They do it through a distributed network, So distributed networks, you know, we talked about it again, it's kind of similar to

peer to peer networks. Well in in in a way it's it's a lot of Uh, it has some similarity to the people who are conducting hacker attacks because as we again, another thing that we've talked about on on occasion is how, um it can be difficult to trace exactly where an attack comes from, or it can be possible to spoof another address, say, you know, make it appear as though an attack is coming from a country to go you know, hey, they're the ones doing it, Yeah,

China's attacking us. And then if you were to really look at you might realize that, uh, the attackers came from a totally different country, but they had spoofed it, or they had created a zombie army where they had infected computers within China, So the attacks were coming from China, It's just they were being masterminded from computers that were

in a totally different country. Yeah's the tricky thing about computers is that, uh, it's it's entirely possible for you to be able to cover your tracks or to off you skate where you are working from and make it appear that someone else, some patsy, is really the one in charge of the actual attack. Well, tour works works in a similar fashion to that, basically uses this array

of servers to mask your location. Right. So, so the way the way it works normally with Internet traffic is that if I'm sending, if I'm seeing a request to a server for me to get let's just say a web page back, then that request is going to go out and pack it across the Internet with a header and a footer that gives the information about where it's going.

And then once it gets to where it is, it's the request gets received, the server response sends it back my way, not necessarily in the same pathway that my request went, uh, and then it gets to my machine. My machine puts it all together, and then I can look at the web page the Tour Each jump from one UH server to one machine to another is is unique, and it's it's unique to that like the the information is unique to that jump. So that UH, when I send my request out, it goes to a server on

the Tour network. Server on the Tour network doesn't know that it needs to go to the specific destination. It just knows that it has to send this request to a second server it's that's in the Tour network. So that request goes to that server within the Tour network. Now that that server doesn't know where the request originated from.

It knows that it got it from server number one, but it doesn't know who sent the request to server number one, Right, So server number two is like, all right, well, server number one sent this to me, I'm supposed to send it to server number three, goes to server number three. Server number three doesn't know about server number one. It just knows that a request came from server number two. It definitely doesn't know about you who sent the request

in the first place. And it does this all the way across the network until it gets to where it's supposed to be, and then it comes back in that same pathway and there's no way for any individual machine to know what the origin point or destination point was except for the ones that were immediately adjacent to those points. So it protects your identity because you can't track where

this this traffic is going. There's no way to figure it out because uh, these machines are It's if you were the only one using this network, sure, then I'd be able to tell. But with all the information passing across, it's impossible to say who sent what when and where this information was going and what was inside it. Now the tour requires that you uh install some software on your computer, which basically you know, uses your browser. You can turn it on and off, um, which is nice

in a way. I did test it just to play around with a couple of years ago, UM, because I thought it was kind of interesting and and uh, you know how it how it masks your identity. So I found I found out two things. One uh, you know, I tried it, and then I went to see where it said I was coming from, you know, I did, Uh, I traced the route and it said I was in Sweden, which is really great, except it made me hungry for Lincoln berries. And then the next thing I was thinking meatballs.

Yeah I knew, and well that's you know cliche. Um. And then, uh, the other thing is it because it is traveling so much farther now, remember we've talked about, UM, you make it a request of the over the internet. The information goes from one computer to the other computer, to the other computer until it gets to the other location, and then it sends it, you know, back back back back until it reaches your machine. When you go through Tour, it does that through a whole lot more computers, and

it slows everything down. So if you want to be private for whatever reason, you can use this software, but it will slow things down. Yeah, just be aware. That's true. That's pretty much true with every security system too. Anytime, anytime you have some encryption or whatever, it's going to affect the speed of the transmission. Um. Yeah, that's that's one of the trade offs you have for for increased security and tour it goes beyond just uh web browsing,

you know. I use that as my example, but it's also used for things like instant messaging or remote logins. You can even create an instance where you're you're creating a rendezvous point, right let's say that you want to create a uh discussion area for some political uh reason, Like let' let's say you're in a potentially dangerous situation overseas, you know, in some in some country where the regime

is is cracking down on internet access and internet communications. UM, you might want to be able to set up a space where you and two or three or more people can have a discussion about something, but not have it pop up in a very popular or publicly visible way

like on Twitter or Facebook. Because even though those tools have been really useful, uh, it's also true that if you use them a lot, then there's the chance that it could either get shut down or that authorities could track you, or that authorities could infiltrate your your communications. So if you're using something that's below the radar, that that's not being broadcast publicly, you have a better chance

of maintaining that security. Well. Tour would let you do things like set up a UM essentially what it would be like a chat room or a forum among a bunch of people that is not visible to anyone else, and it does it through this distributed network way where you the person who creates the instance, lets people know Hey, I'm I'm building this thing on blah blah blah. When they respond, that's until someone responds, there's no rendezvous point,

it doesn't actually create anything. And when that person responds, then one computer, one server on this system is designated as the place where that will live for as long as it needs to. And then uh, the different computers will have to travel via the normal tour method to that particular server in order to interact with it. But this again maintains that level of anonymity that is necessary for situations where you're talking about life and limb, you know, scenarios. So, uh,

it's interesting stuff. And like we said, you know there military has been using this at In fact, tour the onion router was a product of the United States Naval UH branch of the military. It was it was a Navy project that ends up now being used by lots of different UM people, including civilians. But yeah, businesses use it, UH, law enforcement uses it. It's it's been very instrumental in lots of different UH uses other than the one that

everyone thinks about, which is to do something sneaky and criminal. Actually, the only thing that's come out of my navel is lent So that's that's really impressive. Um, I'll tell you what you're gonna give up for lent um. So yeah, But the thing is to note, um, this stuff is on the Internet. And that's actually it's biggest flaw. UM, when you go back to the original sense that Jonathan mentioned earlier, when stuff is just disconnected from the Internet,

that's the easiest way to protect information on the dark web. UM, once you make it connected to that giant network of networks that we've come become accustomed to using. UM, that makes it more findable. It is not a lot of stuff is not completely unfindable. So for example, let's say you have your robots dot text file and you say I don't want my page to be index That doesn't mean it's not there and it can't be accessed by people.

But it might mean that Google and Being and the other Surgeon's is just say okay, well hey, I okay, we'll leave you alone. You don't have to be in our directory. But that doesn't mean it can't be found. Um. You know, you think nothing stopping. Let's say an incredibly popular website could post a link and that would do it. So I mean, you know, it's uh, it would be another step to get to where you needed to go, but it would make that dark website very easy to

access us. Well okay, well okay, so jon and Jonathan and I have created a file a secret file sharing thing, and we're going to let's just make it interesting by making it nefarious. So we're gonna share all these you know, video files and all sorts of other stuff. But we're sharing all the good Star Wars movies, so four or five and six, and all the good Star Trek movies so every other one. So, uh, the thing is Jonathan set up this site and he's emailed me, so he

both died to bring us this site. So they both ends send me the email and okay, so it's between you and me, right, Nobody else knows about it except there's a trace of that email on every server that it touched between you and me. Plus you know, we have I s p s and there should they have that information somewhere and find Now, they're not gonna go looking through all they don't have time to look through all that stuff. There's no way. There's just so much,

but it's there. So if someone else hears about this, like if I become Mr Flappy Gums and I walk outside and I'm like, you should look into that gingivitis. Yeah, I probably should and that, and I say, man, Chris and I we shared episode four Star Wars New Hope eighteen times between the two of us for no apparent reason. Um, and I'm just chatting away. Well, then there's a chance someone goes to someone else, that goes to someone else because someone else. And next thing you know, you've got

authorities looking into it. And they say, all right, well, if we can't find this site directly, what other routes can we look at in order to build a case against these two idiots and and and find out whether or not they're really breaking the law, and if so, how hard we can punish them. And the answer to that,

by the way, is very very hard. Yeah. The chances are that the user agreement that Jonathan and I have between our Internet service providers says that they're not going to give up our information unless the law enforcement authorities come knocking at their door with the warrant saying, hey, we believe that these guys are up to no good uh cough of the records, in which case they're going to say, okay, well, if you've got a legitimate reason

to worry about it, here you go. This is why the various pieces of legislation around the world, really but in the United States are the ones that we were mostly familiar with, things like d M c A, SOAPA, uh and and PIPPA. One of these were such a huge deal because it was one of those things that people were worried the government would get too much authority to be able to snoop in on what you were doing, and whether whether you were doing something wrong or not

wasn't the point. And it was the fact that it was almost like an unreasonable search and seizure kind of approach to information, like it wasn't wasn't necessarily your personal property in the sense of physical things you owned, but it was your information, which is more and more becoming

something like property to people. So yeah, it's uh. And also there the whole SOAPA and PIPA thing was for a while also a concern for people who used tour and that it looked like services like to or could be at threat because of some of the various kinds of laws they are being proposed around the world. Act

as another great example and that's one that's global. It's not the United States, uh, just the United States anyway, And there there's a lot of concern that this kind of legislation is specifically taking aim at services that have legitimate legal and one could argue, uh pro humanity um reasons to use them, and yet it would those would get wiped off just as easily as the piracy stuff. And so the question that a lot of people have is is the damage that's being done by piracy, which,

by the way, is completely impossible to define. We cannot define how much damage piracy does. It's we just can't. It does do damage, Don't get me wrong, we just can't define how much it is. But is that indefinable amount of damage greater than the good that a an anonymous web gives us. And if it is greater than maybe we have to look at wiping it all out. Personally, this is just my own personal opinion, not representative of

anyone else. Uh. I seriously doubt that. But because I look around in the world and how the world is changing, like the events of the Arab Spring, and I think if it weren't for the fact that we have these outlets, then these social changes which in many cases look like they are long overdue, may still not have happened. And and that would be unfortunate for people who are living under oppressive regimes who uh just don't have any opportunities, like like a lot of the rest of the world does.

So now that's my again my own personal opinion. Jonathan's getting off the soapbox. So so, in short, um, you know, the the deep Web, the the invisible Internet. This is this is just information that, um that the average person is not going to be able to find easily. Um. It could be anything from people planning a hate crime or an act of terrorism to uh, people planning their next big venture and they just don't want it to

become public before they can make a sensation. And and and ruin Facebook's day by coming out with something even better to to like fifty hard drives that are holding databases of information that are not relevant unless you put in a specific query. You would not want that searchable on the Internet because it would be totally useless. Yes, yes, so, I mean it's it's a simple definition, but you know there's a lot of two a lot to it. And when you start thinking about it, um, it's there's some

ethical questions involved. But your Iceberg comparison at the top of the show was very apt because again, the stuff that we see on the web is a fraction of what is out there. It's from what I understand, it is a fraction of one percent of the information that

can be accessed over the internet. Um, which when you think about the the billions and trillions of web pages out there, of many of which have MIDI files embedded in them, and scrolling texts and the under construction guy, um, you know that that's amazing to think about because there's a vast amount of information that is accessible to us.

It's pretty phenomenal. But you know, and that's one of just one of those things that that when you think about you step back and you when you think about this vast amount of information, if you just go, wow, us really when you think it's amazing, Yeah, it is is. It's yeah, who would have thought that we have come up with that much stuff? And I have to write about all of it, well some of it, speaking of which I need to get back to that. So let's

wrap up this discussion. Guys. If you have any suggestions for future topics that we should tackle, you should email us our addresses tech Stuff at Discovery dot com, or send us a message via Facebook or Twitter are handled. There is tech Stuff hs W and Chris and I will talk to you again really soon. Be sure to check out our new video podcast, Stuff from the Future. Join how Stuffwork staff as we explore the most promising and perplexing possibilities of tomorrow. The househof Works iPhone app

has arrived. Download it today on iTunes, brought to you by the reinvented two thousand twelve camera. It's ready, are you