TechStuff Classic: How BitTorrent Works

it at its heart illegal? All those sort of questions are the things we ask and try to answer in this episode. Hope you guys enjoy it. To start off, we kind of need to talk about just how you would typically get a file right, how how files are transferred period at all? Under so the normal process by either by by any of the protocols that you to use over the internet, be the FTP or htt P that's file Transfer protocol or Hypertext Transfer Protocol excellent acronyms.

So traditionally, your your computer, which in this scenario we call the client, is going to contact a a host computer a server in order to say, hey, I want that thing exactly ye whether that thing is a web page. So if your web browser can be a client. Yeah, that thing could be a web page. That thing could be a music file, it could be a movie file, it could be uh an email. It all depends on content, right,

that that you want to get. Um. The server would then send the file to your computer according to whichever protocol you are using, right, And it's using the basic set of rules that we all know from the Internet, the idea of everything's in packets. Those packets travel through different routes and then they get reassembled on your computer. But yeah, it's essentially a one to one relationship, right, I'm asking the server to send me something, the server

send something to the client. Done, done, and and you know, and the speed with which this can happen depends on the amount of traffic on the server, the size of the file. Yeah. So let's let's boil this down to an analogy. Alright, So, Lauren, you are a server, I am yea. So now you have entered into the uh the service industry. You are a server in a coffee shop. Okay, you served terrible at that job, alright, but let's let's imagine Lauren, that this is a reality where you're not

terrible at your job. You're actually pretty good. You can serve up to three people simultaneously, so amazing, but only three people and uh, and the amount of time it takes you to produce the order for the customer depends upon what they ordered, like what size of it and how complicated is the drink. Obviously these things will factor into how long it takes you to prepare it. But

you can still handle up to three. But then there's a rush, right like, uh, I don't know, there's some hipster band is in town and everyone needs their coffee, so they all rushed the coffee shop at the same time. And now you've got twenty people asking you to make very complicated drinks, but you can only serve three at a time, and these drinks are getting progressively more complicated

and larger. Things start to slow down, so that person who's you know, seventeen people back feels like they're waiting forever to get a coffee, and they're thinking, what's the big deal. It should only take It's just a coffee, You should just take two minutes. I should be in and out in two minutes, and instead I've been sitting here for fifteen minutes and the lines barely moved. That's

the same thing we experience with this traditional approach. If you're talking about a single server hosting a lot of files, a lot of clients contacting the server for the files, and those files are big, it just means that things can get bottlenecked and slow down. So there were some different approaches to trying to figure out how to make this uh more streamlined, and but one way is to just increase broadband speed. Right, So essentially, if you're talking

about the Internet being a series of tubes, that's it. Bigger, bigger, longer tubes, more tubes, bigger tubes, and more tubes. Yeah, those are your two choices, right. You make them bigger so that you can shove more stuff through the tubes, or you make a lot more of them so that there are a new ways for it to pass through. But you're still kind of ball necked by the the

server itself and how quickly it can respond to requests. Sure, so if this approach is really slow, then one alternative is to distribute a file across a network so that you have lots of different options when you need to get that file. Right, So in this instance, instead of saying that Lauren is the only person who can serve

coffee within a twenty mile radius. And that's why we're all kind of uh, you know, up the creek when we walk into the coffee shop and we see there's fifteen people ahead of Instead, we have a coffee shop on every corner and across the street from each other, and next door to one another. Their coffee shops everywhere. So really I just have to walk into any coffee shop and I can stand in the middle of the street and shout coffee and someone will bring one. Right.

So that's that's kind of the idea of the peer to peer network. In this sense, you have some form of software that allows you to connect to a distributed network where when you participate in this network, you're essentially giving permission to access part of your hard drive. Yeah, you just set aside a little folder on your hard drive that is okay for this program to access, and then everyone else who is running the same program also

has folders set up. Hopefully they have stuff in them, right, Yeah, Because one thing you could do is you can set up this folder and then immediately move stuff out of the folder, and then you're leaching. That's the term, right, right, it's it's considered a faux pas and in sharing circles just because you're you're not putting anything back out into

the community. Right. So with peer to peer, let's say you've got this folder that's designated as a shared folder, and when you're connected to the software, anyone who's also connected the software, when they're searching for a specific file, if your file, if you happen to have that file within your folder, you come up as an option to

connect to download from your computer. Now, the problem with peer to peer, even it solves one issue, right, The one issue is that if you go with the clients server uh model, then you are limited by the server. Like if the server is the only machine out there that has an existing version of this file, then that's the one destination you can go to, and you are

stuck with whatever problems that might have. In peer to peer, you're adding more tubes, but you're not making them any bigger, right, So in other words, like I could you know, if we're going back to our coffee shop, example, and we have these coffee shops everywhere, I can walk to the

coffee shop. That's closest to me, But there's no guarantee that coffee shop won't have the world's slowest server, or you know, like that someone who's just they they are very meticulous about the way they make their coffee, or there might be two or three people in front of me at that coffee shop and I have to wait for them. Anyway, it gives me more options, it does not necessarily mean it will be faster, and the really good software will identify the person with the best connection

who has that file exactly. Yeah, So that way it gives you the best chance of having a smooth transfer. And it's also good to note that this creates more stability when you're trying to get a file. So the other big issue with the old clients server approach is that if that server suffers a problem, if it goes down for some reason, power outage or whatever, then you're stuck. You know, you don't you do not get that file.

And if you were trying to download a big file and it was maybe seventy of the way done and the power goes out, there's a chance that file could get corrupted in the process, and then you have to start all over again, right, which was a real bummer. When peer to pier was was really big back in the early two thousands, like late nineties, um, because you know, it was about that modems that could that could handle broader band than fourteen four it's became available for consumer purchase.

And after that, you know, when when people started getting connected to the ethernet was really when peer to peer happened. Yeah, and then you you were able to you know, one of the downsides is that traditionally, with most Internet service providers, your upload speed is a fraction of what your download speed is. Right. The reason for that is that ages ago, when companies began to offer internet service, they looked at the broadband that they had available, like the bandwidth they

had available to the size of the tubes. Essentially, they're saying, all right, here's here's what we are capable of delivering to our customers. How are we going to determine how much is down link versus uplink? And then they said, well, you know, probably people aren't sending stuff up to the Internet that frequently. They're mostly trying to consume stuff pull stuff down. This was before we were uploading millions of

years worth of YouTube videos every second. Yeah, this was This was back when that was not even a consideration. So the idea was that, well, you know, we can just make it a fraction upload speeds will be a fraction of download because you there's nothing as frustrating really as sitting down. Well this is probably that's that's exaggerating, but it's very frustrating to sit down and try and get at some sort of content online and then just see either a buffering thing or a loading screen and

it just goes on forever. If you're talking about the old old days of the Internet, there's nothing like trying to look at a picture of something and watch it slowly load pixel line by pixel line, and you're just thinking, I have no idea what I'm looking at. I won't know for another forty five minutes. I would I would like the record to state that I just used I just gestured at Jonathan to help you guys understand what he was talking about and exactly how ineffective that was.

That was a great visual representation on an audio podcast. Yeah, so you know that this approach meant that we suddenly had all these other options, this peer to peer approach, and this is the way a lot of those file sharing services back in the day worked because lime Wire, Napster, that kind of thing. Yeah, it's all about let's find people who have the stuff you want and connect this

create a connection directly to them. Yeah, so it almost becomes like a direct phone line in a way, where the connection is between the host computer and your computer, so you can get the file and then of course once you have the file, your computer can become a potential host computer. So now an improvement on that was bit torrents approach, right, and I think it will be really interesting to get into the intricacies of how bit torrent works. But before we do that, let's take a

quick break. Let's get back to the show. Alright, we're back,

and Lauren, you alluded to bit torrent. Now, this was a totally different approach in the sense that it was someone coming up with the idea of how can we make these download speeds faster, not just more reliable so that I have more options, but how can I get the stuff I want faster, particularly if it's a really large file, you know, and if you're if you're like Jonathan said earlier, if your internet connection is maybe not the best and something cuts out somewhere that you won't

lose all of the progress on downloading a file you had already made. So this was the idea of a programmer named Bram Cohen. Now, Bram Cohen had worked for several dot com startups that never really took off, and it just kind of failed over and over, and he really wanted to work on something that worked. That was a big ambition of his understandable And in two thousand one he was also getting really irritated by these problems we're talking about the slow download speed of large files.

He liked the idea of the peer to peer network, but he didn't care for the execution. So in two thousand one he begins work on a new protocol. Now we've used the word protocol several times in this podcast, just to remind you. A protocol really is just a set of rules. It's kind of a kind of seven instructions for a computer to follow. So he designed a protocol that he called BitTorrent. Now, this was an attempt to solve that connection problem um and the speed problem

at the same time. So you know, peer to peer adjusted the connection. He wanted to do speed as well, So how do you do that? Well. His approach was to create this protocol that would allow files to be distributed in pieces, and as you receive a piece, you are also able to upload that piece even before your file finishes downloading. So I might have a file that's at twenty seven percent as it's slowly downloading, that twenty seven is completely available for other people on the network

who are also trying to get that file. You could also get pieces of these files from multiple sources, so as long as everyone had the same file, then you could get, you know, a piece here and a piece there.

So imagine that everyone has at least some part of a five piece puzzle, and one person down the street is giving you pieces uh one through twenty seven, and someone on another side of the street is giving you pieces forty three through and you're just getting You're getting the whole puzzle, but you're getting them from different locations all at the same time. It's like it's like if everyone in that in that coffee house street has a cup of coffee and you just kind of run down

and they all pour a little bit into your cup. Right. Yeah, it's similar to that, except except with less like gross spittle contamination, and also there's a pretty design at the top. I don't know how they managed that while you're running down the street like a crazy person, but you know they're talented, that's all. So he was thinking like this would be the way of solving this problem of slow download speeds. Now, he did not necessarily go about testing

this in a very scientific, scientific PC way. Did you see how he tested Okay, folks, here's how he tested it. He collected a batch of freely available pornographic material and then invited beta testers to see about getting hold of this stuff using the bit torrent protocol. Stay classy, cohen U. But he figured that this was kind of a sure fire away and because it was freely available stuff, it

wasn't like it wasn't copyrighted. He didn't have to worry about running into problems with torrent systems have having fact run into lots of copyright issues. We'll talk about that towards the end of the show. You know, that's that's a that's a totally different discussion. But he he he wanted to try and test it out, and apparently that wasn't considered a terribly auspicious beginning. But he eventually did launch a working version of bit torrent for the general

public in October two thousand two. Now two years later in two thousand four, he had a five person company working on a search engine that would work alongside his protocol. So he designed the protocol, but one of the issues people were running into was how do you find the torrent files which are pointer files. I'll talk about them

a little bit more in a second. How do you find these files that the facilitate the downloading process when you're you're actually using the bit torrent software, and this is via a tracker server. Right, So he created he worked with this five person company, actually five person company worked for him to create the search engine that would look for these torrents to help make this process work more smoothly. Uh. He was also then looked at by

a little company called Valve. Yeah, so Valive. Valve is thinking, you know, we want to do a downloadable distributed network approach for our games. We want to have this ability to deliver games to our users that is fast and reliable and doesn't put a lot of pressure training on our own servers. Right, we don't that way, we don't have to build out a whole data center just to support this this model, this distribution model. So they contacted Cohen and they hired him to work on a platform

that we now call Steam. Cohen was one of the people who helped build that program out, and it was all because he saw what he was doing with bit torrent, and they said, that's the approach we want to take. And in fact, Valve and Blizzard are two companies that use bit torrent for a perfectly legitimate means of distributing their files. It's a good time to just mention there's nothing illegal about torrent files. No, no, it's it's just

a matter of distribution. It's kind of like if you were to say, yeah, but you can get illegal material that way. You can get illegal material through the mail, But you wouldn't say that means we should shut down the postal service. You might say we should shed down the postal service for other reasons, but not for that one.

And same sort of argument you could use for bit torrent, like to say, let's get rid of bit torrent because some people are sharing illegal files, even lots of people sharing illegal files, doesn't mean that the tool itself is wrong, right.

The technology itself can be used for lots of really cool things like yeah, like like letting people like Valve put games up online without having to buy giant servers, or independent artists put their music online, right right, Yeah, And there there are a lot of reasons there their entire you know, whether it's software or video files, music files. You know, there are a lot of reasons why you would want to take this approach. And uh, one of the big ones is that it, like you said, it

takes the strain off the provider. Right. So, if I'm a small business or an artist, an independent artist, I might not have the resources available to me to create a dedicated server where people can come and download stuff, especially if I'm having to pay lots of fees to maintain that. And you know, I mean, you know, for example, if you post some music to your personal website and uh, Neil Gaiman or Will Wheaton link it on Twitter, and

all of a sudden, you're completely overwhelmed. This, by the way, is a complete invitation for both Will Wheaton and Neil Gaiman to tweet about our podcast. We would love that. That would be gorgeous. Um. Yeah, so I wouldn't even know what to do with myself I would probably totally flip out. No, I would totally flip out. I wouldn't probably there, there's no probably there. Yeah, Hey guys, it's

Jonathan again. We're gonna take another quick break and we will be right back to talk more about bit torrent in two thousand five. Already by this time, okay, the bit torrents only three years old from when it officially launched. Even by then, Hollywood had taken notice and was not terribly happy. Yeah, the Motion Picture Association of America and also the Recording Industry Association of America. That's um, the m p A A and the R I double A. Yeah,

r I double A is is like, that's music. Essentially, an m p A is film. So m P A A UH in particular was concerned because the bit torrent protocol did make moving large files much easier, which meant that suddenly people were able to upload and download large files like rips of films. They take a DVD and ripped the film from the DVD and then share it

and you would get illegal copies. Or if you were working on a digital film and you were part of a digital film production and you had access to the file, there were movies that leaked out. UM some famous instances of movies that leaked out before they had even hit the theaters, where people were getting hold of pirated copies, and Hollywood was really taking him a bit torrent even though again, as we said, bit torrents a tool, it's

not like it was specifically facilitating the illegal activity. However, on November twenty two, two thousand five, Bram Cohen had a joint news conference with the chairman of the m p A A. Dan Glickman, and announced that he had agreed to prevent his own bit torrent website from linking to torrents pointing to illegally available movies. Now, before that point, their policy was that if they were alerted to a torrent that linked to those pointing to they would take

it down. So that was there, that was their policy

all all long. But they said, well, now we're just going to make sure that that's much more streamlined, which if you've followed these kind of these kind of cases not just with bit torrent, but with other providers, other platforms like YouTube for example, there are examples of takedown notices that were improperly UM dealt with Yeah, and probably dealt with or improperly submitted people who didn't actually have the rights to something demanding they get taken down, and

then it gets taken down um, and then there then there's some embarrassing ones. I think Microsoft just recently issued a takedown request to itself. Yeah, so sometimes these sort of things can end up being kind of embarrassing. But he was saying that we want to you know, we don't want to up so um. Today you can find bit torrent on lots of different platforms. In fact, bit torrent today allows you to do things like produce content

and then distribute it. So it's design more now to help, like I said, those small businesses, the independent artists to create content, and then even to to enable it so it'll perform properly on things like smartphones, tablets, consoles, creating a huge distribution network that people wouldn't have had access to, you know, ten years ago. So there are many reasons why it's a very useful tool. So let's talk a little bit about how it actually does what it does now.

First of all, it's open source, which kind of tells you that Cohen wasn't necessarily looking to create like a a h multibillion dollar. This wasn't a commercial venture. No, he wanted he really believed in this. And open source means you can see the source code. You can go and get the source code for the bit torrent, play around with it. Yeah, you can change it. You can make your own product based upon it. It's open source um.

And then there are sites that how torrent files. Torrent files are not They don't have any material in them other than pointer information that will point a your software to the right destinations that will have the actual file you're looking for. It's it's a little bit like that that like a protocol that we were talking about, but just in a in a file formata. It's and it's kindlling meta data. It's it's information about the file you want,

the information mainly being where you can get it. Like it's it's allowing the software that you have on your computer and when when you when you run a program to uh to to go go out and find the little pieces of this file across the Internet, across the swarm of computers that contain it. That's good. Yeah. So yeah, your basic computer that has you know, the file, the full file on it, you would call it a seed.

It's a seed computer. UM, and then the swarm is all the different computers that connect to the network that have some portion of that file and are actively downloading and or uploading that file, that are also running this this torrent sideway. Right, everything has to be working running the software at the same time. If you stop running the software on your computer, you disconnect from this network. You're still on the Internet, but you're not part of

the bit torrent network anymore. So, Yeah, the torrent file just kind of points the software in the right direction so you can get these pieces. It gives them the software the information it needs to identify and pull those those pieces of file into your computer. Classically, this is all organized by a central server called a tracker like I mentioned earlier. Um, these these days that's it's it's a little bit um has a little more complicated than that.

But yeah, your classic bit torrent has a tracker server that kind of acts like a traffic driver, like it's it's the one that's making sure everyone is going to the right place. Um. And you're so you've got your seed, you get your swarm. You are constantly downloading as long as you're connected to this or downloading end upling as long as you're connected to this, uh, this network. The cool thing is with bit torrent that your download speeds

depend upon your participation within the network. Right. You get a rank based on how many files you are allowing the system to upload from you, and and your and your actual upload speed as well, So those two things factor in. So if you are being you know, if you're altruistic and you're sharing a lot, then you can also download faster, right because your rank goes up, and that means that your download speed's improve and so you don't have to wait eight hours to get you know,

that music file you wanted to get. It will download in a matter of minutes maybe or or faster, depending upon your broadband speed and the speed of course of the various computers that are hosting pieces of that file. But it does mean that you go much faster than you would with your traditional peer to peer or client server relationship kind of a protocol. You end up getting the pieces of the file you need, and then once you've got all the pieces you you're good to go.

So you don't have to depend upon on computer and its connection to the network. You're depending upon the entire network and anyone who has pieces of that particular file. Now, like we said, there's nothing illegal about this. This is

just a distribution network. So if your file that you're distributing doesn't have any sort of copyright to it, or if it's licensed under something like creative comments, that gives the person who has the file the the ability to distribute it however they like, then there's no issue there. That's completely legal, and in fact, that might even be the intent of that content. You know, the person who made it. Maybe I want this to be shared with as many people as possible. That's why I uploaded it

to the bit torrent network. Uh. However, it does also mean that it can create an environment that allows for illegal sharing in a way that is a lot more difficult to fight than the client server approach, right right, Because it's not located in a single place. You can't go knock on one particular person's door and say you're holding this file. Because it's spread out over so many people that exactly, Okay, it's a lot harder to trace back.

I can't go into Lauren's coffee shop and tell her that she has to stop serving this one type of coffee if everyone around her is serving that same type of coffee. It doesn't do any good, right, right, same sort of thing. Yeah, so this is although I mean, I don't know you could. You can make some arguments that even even the illegal portions of torrenting are beneficial in a way because you know, first of all, it's it's made the industry um create its own legal ways

of distributing files the faster, you know. I I think that the direct streaming on like Netflix and Amazon Instant all that kind of stuff is of direct response. I agree to torrenting. I agree. And it also means that, you know, encouraging the studios to find new ways to get content to two people who who want excellent to buy it. Let me give you my money. They're eager to participate in this. If you make it easy enough and you don't price gouge, then you're not really inspiring piracy.

There's also been several surveys of varying reliability that have suggested that people who pirate stuff also tend to be some of the it's right right, which, you know, people who are torrenting are probably more likely to buy things online than your average consumer anyway, because they're more computer and internet savage to begin with. But but you know, but the number that gets tossed around a lot is from a study by the American Assembly, which is connected

to Columbia University. So it's more or less on the up and up. Yeah, this is that that's a that's a fairly reputable institution. They at least call themselves nonpartisan. They do, they do have a little bit of a liberal slant, I would say, but um, but they found that peer to peer file shared users purchase more music

online than non peer peer us. And then of course there's also the argument that a lot of these organizations make the mp A A in r I double A is that UH stolen property UH directly translates to lost revenue.

And we've seen multiple studies, including studies that were specifically UH funded by the government, you know, the government agencies that looked into this to see how much, how how much in damage is really is caused by piracy and the conclusion is that you cannot come up with a number because you cannot say for certain that someone who pirates something would have purchased that something had they not

had access to the piracy software. And furthermore, that they're not going to go out and purchase it afterwards that they weren't trying before they bought it. Right, So in both cases, you don't know if they went out and bought it anyway, and you don't know if they would have bought it. So without knowing those two factors, you can't say that this actually cost anybody anything. And and you know, it's not like a physical copy of something, where if I walk into a store and a shoplift,

that store is out a physical piece of inventory. When you download a digital file, you're making a copy of something. The original version of that file is still on a person's computer somewhere else, so it's not like they have lost that. So with all of these factors, it really means it's so complicated that we cannot put an actual dollar amount. Not that that has stopped anyone from doing so when creating massive lawsuits against either a company that

creates the software or the users of that software. Absolutely, And we we will talk a little bit about that in a future episode that you'll hear probably in just a couple of days, because I think we're recording get immediately after we finished this one. Yes, we're talking about a specific right, um, My favorite unofficial study. By the way, is an anecdote on the effects of purchasing and and

free downloads. Was when uh Neil Gayman in his publisher, which I believe in this case is HarperCollins, um put up a copy of American Gods for free wo and just just free ebook download. Go do it, people, And I don't think it's up anymore. I might you might not be able to find it. I could have just lied to you tremendously. But they had. They had a free copy at for a certain period of time, and sales increased, definitely increased, like appreciably increased in the immediate

future after it was up. I will say that, um there, you know, And I've told this story several times in the past, so you know I'm comfortable telling it again. Uh the you know, I I definitely was one of the people who downloaded something outside the realm of the law. Because I downloaded a television show that was made in England that was not available in the United States. There was no way for me to purchase it legally. Not

that that justifies illegal behavior. It doesn't. So I'm still in the wrong, you know, I'm still in the wrong for doing it, but I downloaded Space the show that eggar right and Simon peg and and several other people did, and uh, and I loved it, but you know, there was no way for me to lead to get it in the United States at that time as soon as it became available as the day that it came out on DVD and purchase it because I loved it, I wanted to support it, and I wanted to have a

really good copy of it. Um. That's another reason that things that like the studio backed stuff is getting more popular because the quality tends to be better and you don't have to worry about like malware is usually unless the d r M is also malware. Sony, I'm looking at you. Um it's been much better recently, by the way, but you know that that CD thing still sticks in

my craw. I think even classically though that I mean, you know, this conversation is reminding me of you know, how used to go to NERD conventions and they would have all of these illegal Hong Kong VHS tapes of various UM did Battle Royal that way, But I bought that as soon as it came available to and and I think that similarly that really that kind of underground

black market VHS sort of thing. Aside from inciting nostalgia, people of a certain age really pushed movie companies to say, oh, hey, there's there's a reason, like there's a reason for us to publish this in America or in other countries outside of their original origin. People love this, there is a market for it. We can you know, if we provide it and we price it properly, we will make money. Like that's that's the lesson that a lot of these

companies have learned. Uh. And people can argue that, you know, these are things that various industries have had to learn uh in a staggered amount, So like the music industry learned it first, and now the book industry also because with the books that definitely raised it. But as broadband speeds have improved for the common users. So so it's interesting. I mean, and again, you know, BitTorrent is just a

means of distribution. You don't necessarily have to have it be something where you're pirrating movies and tv um um. There there is a little bit of a fight back.

As of April of teen, McAfee head patented a system that identifies pirated content and can prevent users from downloading it by UM either blocking it entirely depending on how you set up the software UM it's it's through side advisor, So depending on how you set up side advisor, it would either block the software entirely UM or block the download entirely I'm sorry, or just give you a little pop up window that says, Hey, it looks like you're

trying to use a torrent file that is torrenting illegally copied material. Do you really want to do that? Maybe here's some legal alternatives that you could use to download that that on the up and up. And that wraps up this classic episode of text Stuff how bit torrent works. Hope you guys enjoyed it. If you have any suggestions for future topics I should cover on the show, reach out to me on Twitter or Facebook. The handle for both of those is text Stuff hs W and I'll

talk to you again really soon. Text Stuff is an I Heart Radio production. For more podcasts from I Heart Radio, visit the I Heart Radio app, Apple Podcasts, or wherever you listen to your favorite shows.