How the Wii Remote Works

introduce me? Second? Because otherwise I'd be going hi? And Jonathan Strickland's here and Chris Polette, Yes, we we had a listener asked why I was always listed Second. It just because we kind of settled on this routine. And uh, once you get a routine, it makes you feel nice and comfortable, you don't really want to leave that. And so when I tried just a minute ago, I got lots and lots of giggles. Yeah, yeah, because it just didn't work. Um, rest assured, faithful listeners, I am not

offended or hurt in any way that I am listed. Second. Uh, the fact that I pretty much chatter all over Chris whenever he tries to say anything. Uh, I think he needs I think he needs top billing all right, So let's let's get into the topic. Yes we should, Yes we will. What we're gonna talk about today comes to

us courtesy of a little listener mail. This listener mail comes from Jackie, and Jackie had a nice mail message was fairly long, so I'm just gonna cut it to the chase here and says I had a topic that was wondering about how does the WE remote actually work. I know it has something to do with the sensor, but that's it. Looking ahead the future podcast, I can't see any that appeared to cover this topic and think that it would make an interesting podcast. Jackie, we agree.

So we're going to talk about how the WE remote works. And we've talked a little bit about the Wei remote in the past when we were chatting about the you know, the different video game interfaces, but we really wanted to dig in and talk about what it is that makes the WE remote tick, how how it works, and UM kind of some of the neat things that people are doing with the Wei remote that perhaps Nintendo didn't really intend. Yeah, that's uh, it's funny because the pieces that make up

the WE remote. UM. Actually one of the things that people talked about a lot when the Nintendo uh we first hit the shelves was the fact that, um, you know, people who who were detracting from the machine said, well, you know, they didn't use any of the high tech, custom, cutting edge components that some of the other manufacturers use. But that sort of works in uh well, it works in Nintendo's favor in one regard because it keeps the cost of the console down here. But um, as it

turns out, these components are also prized by hackers. People like to take stuff apart and do other things with it. I've seen many projects, uh cataloged in the pages of Make MAGAZINEEN and YouTube and some of the other places where people have messed around with it. Um. And it's really the WE remote is a hodgepodge of different parts um that basically take your movements and interpret them in a way that the console can understand and allow you

to play the games. Right. So let's kind of talk about the different parts that are inside your WE remote, assuming you have one. Okay, all right, So if you were to take your WE remote apart, here's where we say, do not take your WE remote apart. People have done this for you. You don't need to do it yourself, and if you do do it, chances are you won't get it back together again. But if you were to take it apart first, you would notice that it's held

together by some pretty wacky screws. The screws are proprietary, it's a Nintendo design, um, and it's designed in such a way. It's kind of this weird sort of triangular, three pronged thing. Uh. It's designed in such a way so that it is not easy to remove those screws, although you can find and screwdrivers out there, especially for Nintendo repair kits that that do have the right um, the right head on the screwdriver so that you can remove them. Uh. It's held together by about four screws

and a couple of clips. And once you get that apart, that's it. That's the only thing holding this this bob together. And we're talking about the main we remote here, not the nunchuck or anything like that, right, not the motion plus right, not the motion plus either. Right. So once you get that apart, you would be essentially looking at a plastic casing, two halves of a plastic casing, and a circuit board with lots of a little you know,

microprocessors attached to it. You'd see a speaker there, you'd see some buttons. Um. You would also see a little motor that would be attached to one to the upper side of the we remote that motor. I'll just get that out of the way first. That's pretty simple. That's the motor that creates the rumble feel that you get whenever you have one of those, uh, the force feedback moments. Yeah, similar to the kind that you'd find in a cell

phone that you used for the vibrant mode. Yeah, it's essentially a little off center rotating cylinder in there that rotates and that's what causes the rumble motion. And it pretty much just has an on and an off mode. That's about it. Though. You can make it feel like more of a of a vibration sensation if you turn it on and off at a at certain frequencies. That can actually make it feel like it's a more intense or less intense rumbling. But like we said, it's it's

it's a bit. It's a binary function. It's either a zero or a one on or off. Um. Actually, uh, Jonathan and I were when we were doing research for this, we found a great article in the I e e ES Pervasive Technology magazine from Johnny Chung Lee. Uh, I can highly recommend it if you're really interested in hacking this. But this is where we found a lot of our information. And he gets into lots and lots of detail. Uh. He was a doctoral student at Carnegie Mellon University and

is now part of Microsoft's team. I believe he has a pretty cool blog out there too. Um. But yeah, so there we we got some really really good details on this. And should we get into the individual components. I want to talk about those. Um, you know, the first thing, uh, you know a lot of people think about is that three axis accelerometer that's inside the machine and analog devices. Is the company that created that yep, yep.

And uh, basically you'd find these in other all kinds of other technologies to they're they're pretty common these days. Cell phone and you know the iPod and iPhone and iPad. These are the sort of these are the sort of sensors that allow the devices to know which way they are oriented. So like for instance, an iPhone or an iPod touch it would tell that it's whether it's in

landscape or or portrait mode. So the screen would flip accordingly, same sort of thing, except in this case this the accelerometer is helping the um the we uh figure out what the orientation is of the WE remote itself, so the pitch, the yaw, the role um as you move it around. This is y'all, it's yeah, it's one of the This is one of the ways that the WE detect how you're holding the remote and thus can interpret your actions within a game or you know, a menu

system or whatever. It's just one of the ways, but it's it's the one that that got a lot of

press early on. And uh, I read one review where it was kind of interesting where they're talking about taking a part that WE remote and finding just a tiny bit of disappointment that there weren't actual tiny little gyroscopes in there to indicate the because the gyroscope is a you know, it's a good old way of figuring out the um the orientation of a of any sort of surface, right, I mean, that's kind of what we what NASA uses in a lot of it's the early spacecraft where gyroscopes

too to be able to make adjustments to the pitch and y'all. But no, no, this is just a little like tonic chip that essentially does the same thing it uses. You know, it detects different changes in fields. So um, but it's there is no actual little moving gyroscope inside your Wii remote. No gyroscopes, no jack's, no tinker toys, you know. Um, sorry I got out track there. But there is an I R camera, an infrared camera manufactured by Pixart Imaging. Um that's got a multi object tracking

engine in it. It can even it can keep track of four simultaneous infrared light sources. You know. This is what allows you to do lots of really cool hacking tricks, which will give into a little bit later. But if you're like me, you might be a little surprised. I was thinking that would be an I R sensor right, not not not a or not not a sense I'm sorry, and I are emitter, and that the sensor itself would be on your television. You know that you have a

sensor bar that you put above the TV. Because it turns out it doesn't really sense, no, it it's emitting. Yes, Um, so it turns out that the uh that it's swapped around, right, I thought, because when you think of like a remote control. When you press a button, you think, oh, I'm sending a signal to the television or to the you know whatever, the cable box, and that the cable box is a receiver, and so that's how that's working. No, it's the other

way around with the WE remote. The the remote itself is the sensor, and the sensor bar is actually the transceiver. It's transmitter, actually not transceiver, but a transmitter UM. So you've got at the end of your Wi remote this sensor that detects uh infrared UM signals. And that's again we'll get into it, but that's kind of the secret behind we hacks and why the Wei remote has become very, uh, a very useful tool for hackers. Uh. Let's see what else.

What else? It's got four little blue LEDs in there tell you whether you're player one, two, three, or four. The buttons, of course, are on little sensors. So every time we press a button, that activates the sensor and and the uh. When you get down to the point of taking a Wei remote apart um, there are a lot of buttons there. It's twelve buttons total, and four of those would be the direction pad. A lot of people just think of the direction pad as a single unit.

But yeah, technically it's four buttons. Um and uh, well, it's got an expansion poured, it's got it's got the port for the nunchuck or we motion. Plus it's a proprietary six pin plug that's Nintendo's own memory. Yes, it does have memory. It's got five point five kilobytes of on board flash memory, which is good for storing your ME if you're going to take your ME with you as you go to a friend's house to play. Um, just a little bit of information there. It doesn't it

doesn't store lots and lots of data, all right. There's the speaker you mentioned that earlier. Yes, so that you actually can get sound out of the Wei remote. So if you're playing something like a Star Wars game and you fire up that lightsaber, you get the nice little wall wow noises coming straight from your controller and not from necessarily just the TV. UM and uh. It has

a Bluetooth chip. Yes, it does. Is another reason why hackers like it so much, because it makes it easy to connect to various PC programs that have a Bluetooth enabled UM uh component. Yeah, it's a it's the Broadcom two chip UM. It's uh, it's not a dent human interface design compliant, which means you might if you are going to try hacking the Wei remote yourself, UM, you're gonna have you might have some difficulty connecting it to

a computer. Although according to what Lee said, it's probably the hardest part of actually uh doing a lot of work with it is getting it to connect with a particular computer. So what it depends on the computer, I think, and whether or not you can you can detect it easily or not. But but yeah, it does. That does help out in the hacking department. I forgot about one other component that you would notice if you were looking

at the circuit board, a capacitor. I am, there's actually quite large capacitor on the end opposite of the UH the I R sensor UM. The capacitor is is probably probably provides some power to things like the accessories that you can plug into the base of the Wii remote for example, Again the nun chuck um, which on its own does not have any battery power. It's it draws all its power from or doesn't have battery power. And note I can't remember it's been so long since I've

used my Wii. Um, what the way were on the nun chuck. The nunchuck doesn't have a battery on board. I didn't think it did. But as soon as I said, I was like, whoa wait, if I'm wrong about that, people are going to write in the will it's been so this shows how long it's been since I've used my Wii remote. Well, I do have in my notes the connector is a proprietary Oh I said that, but it does provide three point three volts of power and four hurts of twelve C serial communication. Okay, so that's

where the nunchuck is drawing its energy from. The battery provides the power to the capacitory. Topacitory can provide power to the nunchuck. Um. Also, I mean the dun check is gonna draw power from the we remote batteries as well. Yeah. And if you are using batteries, um, instead of a rechargeable battery of some type, um, you know, like they have the third party and uh um the battery packs

that you can actually plug in battery or charging stations. Um. But if you were using just a standard pair of double A batteries, you're gonna get about twenty to forty hours of battery life. Yeah, that's not that's not terrible. Now when you think about it, I mean considering you know, the sort of functions that you're using this this WE remote for now, if you're using the Wii remote for something static, you're using the Wii remote as a detector

um the way a lot of hacking projects use it. Uh, it probably will last longer than if you were to, you know, use it as a full um video game controller, because then you're activating the accelerometer, the rumble pack, all that kind of stuff, and that's going to drain more power than if it's just being used as a sensor. Now where should we go from here? Well, if you think now, that's I mean, that is exactly how it works. You're talking about again, the sensor bar has the LED

lights inside it, so it's the transmitter, not really a sensor. Oh. We we should point out the function of the censor bar because somebody might say, hey, you know it's got all that stuff inside the controller, why do you need the sensor bar. The censor bar helps orient the WE remote and the WE so the WE knows pretty much

where the player is. Um. You know, if you start anybody who has a WE and especially in someone who has a someone like me as a pretty small living room, if you you'll find that you step if you step close to the TV suddenly or we may go wait a minute, I don't wear Hey, where did you go? Well, it's the the center bar and the camera on the WE remote are basically telling the WE roughly where you are, and if you make a sudden shift forward or backward suddenly,

that we might take a minute to reorient you. And that's that's what that lag time is, is the censor bar and I our camera basically figuring out what happened, and the WE motion plus takes care of a lot of that. That does help. That helped quite a bit so that it could give more accuracy as far as

depth goes. And uh so, yeah, the censor bar is actually got lights inside it ELI d Infra red lights, so not we're not talking visible lights, it's outside the visible spectrum, but they're constantly sending light out and the censor at the end of the WE remote picks up that light and that's what and through the Bluetooth chip can transmit to the wii UM the orientation of the remote based upon things like the accelerometer and how much

of the light it's sensing from the sensor bar. Now, if you were to decide to hack this, you're looking mostly at the Wei remote as your your device. You know, you don't even need the we anymore if you want to. If you're if you're talking about a hack job where you want to use the the WE remote as a sensor, what you would need is some sort of of either reflective tape or an l e D i R light infra red light, so an infrared l e ED. We're not talking about your average l d's that light up

red or green or blue or whatever. It would be lighting up in the infrared spectrum. Now, I've seen some really cool applications for this. The one in the article was actually pretty neat. Where was talking about using uh, the l and an l e D marker. So you're it'll looks like a marker, like you know, like a like a sharpie, Yes, but it has an l e ED infrared light at the end of it. And then you could project on a screen a um projected a white board and you could write on the white board

with the l e ED light. Very cool. That's pretty nifty. But the one that I really like was an a Wee Ferreman. Yeah. Interesting, it was a fairman where the Fairman, if you aren't familiar, is h this wacky musical instrument that um you usually play by having a you have a pair of sensors really um and as you get closer and further away from him, you can adjust the

pitch and volume of a sound. And it's that weird like wah wah, I'm noisy here in the old like nineteen fifties b science fiction movies and almost in in some ways it reminds me of a saw people playing a saw, right, but you can identify the I mean

it sounds electric. Yeah, there is a difference in the the eerieness of it sort of reminds me of saw what it's like, and our keen eared listeners will pick up on the Stephen King reference I just threw in there, but yeah, it's the idea of the traditional thereman is by moving your hands further closer, further away, or up and down these various u UM bars, you can make

uh this weird sound. Well, it's kind of a similar idea with the the the Nintendo we hack, the we remote acts as the sensor and you use a pair

of gloves. Where the guy who did this hack, he used a pair of gloves and he put an L E D light at the tip of the forefinger of each of the gloves, all right, and then by orienting his hands, he could move one L E D up and that would be the the I think it was the volume, and he could move the other led closer or further away from the center that WE remote, and

that ended up being the pitch. And so he created a virtual thereman, which if you've ever looked into the plans for building a thereman, these things are fiendishly complicated. They're not easy to build. Um. I actually know a guy. I know a guy who did build one, and it took him quite a while to do it because it's just it's it's a pretty complicated build. Um. But by using some software the that I think the guy actually wrote the software who created the hack, And this I

saw on YouTube. I just did a YouTube where I did a I think I was searching Theraman's at the time and I saw this cool WE remote hack. But using the WE remote as the sensor, it's the one that's picking up where the led lights are coming from UM and then running that through this software, he was able to create the Freman UM, the virtual Freman. Yeah. It's matter of fact, I've seen, uh seeing quite a

few we instrument type hacks. UM. The guy who had created this entire percussion uh ensemble where different kinds of instruments where he had it was really involved lots and lots of servos uh and other little robotic things used to strike the drums, you know. And UM he had it set up where he could use the Wii remote to control what would be played at what time, and it was very very fascinating us. Uh quite a few

videos on YouTube like that. UM. And uh, you know, I'm curious, why would somebody bother if you, if you these parts are basically off the shelf parts, why would you say that people actually bother taking the time to hack the Wii remote instead of going to get an accelerometer and you know the l ed s it's already

made for you. That's a big thing. So it's it's essentially like uh like going to for example, the maker shed and getting an argue know board, and you know it's already You've got the components that you need to do it, and all you have to do is pull it apart and do what you're going to do with it. That's part of it, I think. But I think another part. I mean, you've already, like I said, You've already got the components put together. You've got the Bluetooth component, which

is a big part of it. UM, You've got the the infrared sensor there, and it's already all wired together in a circuit board UM. And it's in a very compact format, and it's cheap. You know, it's already put together in a in a relatively cheap way. We've talked a little bit about some industries, like say, the virtual reality industry, although it's most people tend to call it

virtual environments now or they call it something totally different. Well, it's hard to get funding for virtual reality and virtual environment projects. It's the the public, uh, perception of virtual reality was not accurate back when the term really took off in the nineties, right because we all heard about virtual reality, and we all heard that we were going

to be living in this virtual space. And once people found out what what, how far along virtual reality actually was, the bottom dropped out of it because it did not live up to what our expectations were because a few people kind of overhyped it, and as a result, those projects lost a lot of support and it became more and more difficult to get funding for those projects. Even though the work was still very much, very important, it

just wasn't as far along as everyone thought it was. Well, industries like the virtual reality industry started turning towards the video game industry for a lot of their UM, a lot of their their equipment. They just cannibalize whatever they could the video game industry, and the we mote is a good example of that. Um You can use the we mote to UH to create a kind of virtual environment.

One of a good example actually that's in that article that you mentioned, was a pair of glasses that allows you to create a parallax view of virtual environment where you would put essentially you have a pair of glasses and on either side of the glasses would be one of these l e ED emitters infrared emitters. You use the Wii remote as the sensor. You would build a virtual environment, and you would link the we remote to

the virtual environment. You put on a pair of glasses, so you're viewing the virtual environment, and as you turn your head, because the Wii remote catches the L E D s and finds the different orientation, it would change your view of the virtual environment as you turn your head, so it's like you're looking into a three D environment.

So you know, as you look at a flat screen, but you turn your head left or right, and because the orientation of the lights is different with the Wii remote, it changes your view on the screen and it looks like a three D environm Man, it's pretty cool. I've actually seen that kind of uh displayed once before. It's a little hard to explain. There are videos on YouTube of this this uh, this effect. Um. It's a pretty cool,

very simple virtual reality implementation. But it's the sort of thing that that the VR experts really get excited about because it's within, it's you know, much more affordable, and it's really accessible. It's not like there's only one of these devices in the world. You can just run down to an electronics store and buy one, you know, by by we remote off the shelf, and then with some software and a little hacking that you're done. You've got

yourself a virtual virtual reality machine. Yeah. Yeah, And I was thinking too that the the other factor that plays into this is the hacking factor, which is the idea that you're taking something that already exists apart and doing something with it that was not originally in tended. What's just adds to the fun of doing that. That's the

classic hacking ethos. Yes, I mean that goes way back before computer is even to like hacking the telephone system, where you you create ways of making it do things that it wasn't meant to do. We should totally do a podcast on that, all about phone freaks. Yeah, we could do that. That's a good example, all right, write it down, Okay. In a future podcast we will talk about phone freaking. Okay, Steve Jobs will feature heavily in that, I'm sure, and we can get on the phone. That

would be awesome. Um well, we've already talked about project at all in previous podcasts, but we haven't really gotten into the latest developments on the Sony move, like, for example, we actually have a name for it now. Um, we used to call it just the wand yeah, well it looks sort of like a wand it's uh, it's got a glowing ball on one end. But it will officially make its debut this fall. Um, the Kid's gonna cost a hundred dollars. It's got it will have a game

in it and a camera. It also uses a camera to help capture movement more like more like New Tall than We, which is sort of backwards from that, right, but the camera is on the device end, like the television end, rather than on the the controller end. Yeah, but people seem, especially the PlayStation three fans, the hardcore fans, seem pretty excited about it. Um you know, going to be able to do motion controller work with a lot of the games. UM. And it too will also have

another UM sub controller for the other hand. UM, so you know you'll you'll be able to use both hands, not just the one right when you're playing PlayStation three games. But interesting how this all has moved so rapidly and has become such an important part of console gaming and even even computer gaming because now that people are doing that and using um, you know, the motion sensing capabilities

of these different components. Yeah, I think so and Microsoft both, I mean they both clearly had these projects in development before the The We debut, because I mean these are long term projects or not something that you you know, just turn around very quickly. However, I think that the success, especially the early success of the we probably put them on the fast track on in Sony and Microsoft's case.

One of the interesting things I noticed about Project Natal I just did a quick look to see if you know, more information about it, just sort of the stuff that we figured out since the unveiling at E three last year. UM pretty quiet, really, you know, did they at least

did the consumer. But the one of the interesting things I saw was read up on on how the brain interprets the you know, it's what Microsoft refers to as the brain Natal's brain, not not the human brain, but how the natal brain interprets motion and uh and your posture in order to transfer that into a a in game action and uh. Essentially, what it does is when the cameras pointing at you, Natal software is scanning your body and uh and matching it with a database of

virtual skeletons. Yeah. So yeah, there's this whole database of different skeletons that it it has within its memory, and it kind of matches you to the one that it thinks it fits you the best. It looks at your form and then says, okay, Uh, this point here, that's a hand. That point over there that's the other hand. This must be an elbow, that must be a shoulder. There's like thirty different points that it matches up to a virtual skeleton. And uh, because it doesn't automatically know.

It's not like the camera magically knows that your hand is your hand. It's interpreting it based upon previous information. And uh, the more you use the natal feature, the more it kind of figures out, you know, oh, well this I thought that was his hand, turns out this is actually his hand, right, Because I mean, it doesn't know automatically right now, it's not not doesn't magically understand,

you know. And we're all we're all different shapes and sizes, so and depending on what you're wearing, that might also kind of confuse it, Like a shirt might make it think that your elbow is in one location when in reality it's in another. And when it sees that you've bend a certain way, it's like, well, either his arm is broken or his elbows actually over here. So it looks at your body, it compares it to its database of virtual skeletons. It matches you up, and it does

this in just a couple of milliseconds um. And it has to because if it didn't, then the games wouldn't work because it does this thirty times a second, because as you move, it has to adjust your the skeleton. That makes sense, It's just right. Yeah, it's really really fast. Even so, I've heard people who have used the NATAL system, or at least the the prototypes, say that there is a perceptible lag. It's not a big surprise because it turns out that we can perceive a lag of just

a few milliseconds um. You have to drop that down as low as possible if you want to avoid like within virtual reality, you gotta drop it down because if you don't, you get virtual seasickness because as you turn your head and then the virtual environment changes to reflect that. If it's not seamless, it makes you feel a little disoriented, and you can actually get motion sickness from virtual reality

if it's not matched up close enough. The same thing when at all, If you see an action on the screen and you react to it, if it's not really fast, then you're gonna you know, you're gonna get killed in the game. You see something coming at you and you react and then in turn, the game has to react to your reaction. If it's not really seamless, than it's gonna be a very frustrating gaming experience. Yeah, it's it ends up being a little more of a lag than

say that we remote. It's it's not as instant a feedback loop as whether we remote. So okay, then well the moment, that's all I got too. So I guess this will take us to a little listener mail. This listener mail comes from Andrew. And Andrew says, Hey, guys, I'll listen to your amateur radio episode last night, and I'm just curious what is the purpose of having licenses for different radio frequencies and such. Is it just a way for the government to make some extra cash or

is it dangerous to the environment or people. I'm just curious, whine matters whether someone transmits radio signals without a license. What harm is being done besides the government not making money? Thanks guys, Well, I wrote back to Andrew already, but I thought this might be an interesting thing for our listeners to hear. Um. It's really a matter of of

just uh managing all those different radio frequencies. If you're broadcasting over a certain radio frequency within a particular geographic location. Let's say it's a radio frequency that doesn't really go that far but within your town. Um, other people can't really broadcast on that frequency. Um or if someone already is broadcasting on the frequency, you're essentially creating interference for that person because you can only really transmit or receive

over one frequency uh at a at a time. I may you can receive from one frequency and transmit over a different one, and that's how you can have a conversation with somebody. But it's really managing that making sure that you don't have this cacophony of noises and interference

uh happen over any particular frequency. Yeah, we talked about how ham radio got its name and for the people who are basically talking over everyone else, you know, they were the hands they were doing and and so the license is really so that you understand this kind of stuff. It's for the government to know that you know what you're doing and that you're not gonna end up, say broadcasting over a dedicated bandwidth for something like radio frequencies

or television frequencies or emergency frequencies. And that was the point also of having the network, right, So that's really the whole point of the licensing is really to make sure that you know what you're doing and you're not gonna end up causing trouble for everybody else. So it's not just an idea of you know, not some sort of money making scheme, because I'm pretty sure the government is not making money hand over fist with licensing fees.

But um, good question. If any of you have any questions for us, write us our email addresses tex stuff at how stuff works dot com And we have an article on how stuff works dot com about the Nintendo we so if you want to learn more about it, I recommend you go there and check it out and we will talk to you again really soon for moral

this and thousands of other topics. Does it how stuff works dot com and be sure to check out the new tech stuff blog now on the house stuff Works homepage, brought to you by the reinvented two thousand twelve camera. It's ready, are you