Did Minority Report Shape User Interfaces?

eternity now, well, especially when that week includes dragon Con. Yes, must just got back from and they're a little bit crazy. You might you might be able to detect My voice is not what it usually is because of dragon Con. Yeah, we we both sound like more like Tom Waits than usual. That's right. Yes, the uh, the hosts have not been drinking, just the microphones. Okay, is the future of dragons? You know? Uh? You would? Yeah, I'm pretty sure, but but no, what

was your what was your Yes? We did, we did consumer technology. I do remember this and whether or not it will be as awkward in the future as it is now, and how that's portrayed in science fiction films, right, that maybe the idea that in science fiction and the movies. When we see the technology of the future, it's just too nice, it just works too well, it's unrealistic, right.

And one of the touchstones we referenced there was the movie Minority Report, which very often comes up in discussions about future consumer technology because, let's face it, it has informed a lot of the post two thousand aesthetic about future technology. Right. In fact, you could say that it was predictive of many different technologies. Uh, we're specifically going to be talking about user interfaces today, but that's not the only one in Minority Report that has become kind

of a if not a household term and emerging technology. Sure. Some of the other things include their interesting strange weapons in the movie. They've got these little non lethal sticks they use the jet packs. They've got robotic cars, self driving cars that sort of hook onto these weird tracks. They have the big data and Internet of things the world that responds to you as you move through it. So,

for example, the personalized advertisements, that kind of thing. These are all this is all stuff that we're seeing in some form or another today. Yeah, and all of this came out in a movie that premiered in two thousand

and two. That's what like five years before the iPhone. Yes, yeah, so, I mean even those user interfaces that we now like when you when you see John Enderton Tom Cruise's character moving his hands all willy nilly in order to sort through different stuff, and you think that kind of is how we deal with technology today, except we're actually touching a screen most of the time, not you know, putting

on gloves or whatever. Um, you might think this this movie it was incredibly prescient, although if you look into it you realize that there was a lot of work going on behind the scenes. But we'll get to that in a minute. Well, yeah, let's start with talking about Minority Report itself. So we don't need to discuss the plot of the movie too much because maybe even if you haven't seen it, it's something you should experience as

a surprise for the first time. But it's a sort of futuristic crime related thriller that deals with issues of free will and and things like that. But the technology of the movie is what we're focusing on today, and specifically the technology of how Tom Cruise and other characters in the movie manipulate virtual objects and data. Yes, yeah, we want to look at that and how has that shaped our user interfaces today and what might it be in the future. Okay, so when Tom Cruise sits down

or not sit does not sit down? When Tom Cruise stands up at his computer, yes, movie, what's it look like? What's going on? So it so a big semi circular screen that he's working on, right, that's it's kind of giving him a panoramic view of lots and lots of data all at once. Well, there's one thing I'd like to add to that, which is that it's a clear screen. Something I'm going to focus on later. But yes, a

big semicircular, clear, transparent pain of glass. And what happens on that glass, Well, that's where all this data gets projected, including video, text, lots of different stuff going on at once. And in order to sort through this, instead of using say a keyboard and mouse, he dawns these kind of electronic gloves, right, and those gloves are acting as his

link to the data that's on the screen. Moving his hands around, he can select specific pieces of data, he can resize it, he can swipe it out of the way if he's done with it. One of the things it's often compared to is that it looks supposedly like he's conducting an orchestra. I think that's even something the director, Steven Spielberg was going for when they went into it.

He wanted it to look like Tom Cruise was standing there conducting all of his data as you would the different instruments in an orchestra as they come in and out, and and he's so he's he's got his arms going all over the place. It's really all about how would you physically manipulate the thing you're seeing on the screen, knowing that you can't actually touch it, right, you know, you can't touch the video that's playing on the screen, So how would you if you wanted to make it bigger?

How would you make it bigger? And you think, well, if it was like, you know, silly putty something like that, that I could physically have my hands it stretch it out. Yea, yeah, you take the corners an you kind of stretch yep. Uh. So some other screens in the movie, one thing I've noticed is that they're also often clear screens, so they're transparent, but they have data facing one way on them and their multi touch displays right, so they are touch screens

like your iPhone would be. But many historical touch screens could only deal with one contact point at a time, or maybe one or two contact points at a time, but but not interactively with each other. It was more like I can press this button or I can press this button kind of issue. Yeah, Um, minority report shows screens and hovering touch based interfaces that accept multiple points of contact simultaneously, which we actually do have in super

technology now. Yeah, And the back in the day, it used to be where if you had a capacitive touch screen, that's the kind where it's detecting. It has a very weak electric field that goes across the screen. When your finger or really anything that's conductive touches against well, it has to be a special stylus for a capacitive but if it if it is a conductive material and it makes contact, then that interrupts the electric field. And in the old days, really you could only get the one

point of contact. And just think of the the think of like an invisible grid that's laying across that screen, and when you press against it, you are interrupting some of the lines on that grid, and that's how the computer knows where you are touching on the display. There are other types of displays that rely on pressure, which originally you could get multi touch displays that did that that that could detect several different uh actual pressure points.

But the problem with those is that it uh they can wear down pretty quickly because you're actually having to push against the screen, it's not just gently touching. And then of course there are other ones that use cameras

to actually detect where your fingers are. Now, these would have to have either a top facing camera, you know, top down camera so you're thinking of like a table, or it would be actually cameras mounted underneath the screen pointing up that are probably using infrared, something that's not going to interfere with us when we're looking at the display, so they can detect where the points of contact just secretly shining into your eyes and making you blind over

many decades, you know. But but on the flip side, you can use all your fingers when you're trying to manipulate datas. I'm just kidding that. I don't think they do know now, but there are lots of different implementations to achieve this particular outcome. Yeah, And so then we also can look at the specific gestures that are involved in the gesture UI in Minority report, and some of them are ones that are very common now, like swiping, or they pinching or stretching for zooming in and out

of images. I mean that's practically universal. So, and it's a little problematic because you have companies that have patented certain gestures and saying that they essentially owned those, and that's caused some tension in the touch Screen World book. Yeah, hold on a second. You can patent the idea that you swipe something to get rid of it. Currently, Yes,

you can. You or you can at least submit a patent to the Yeah, you can submit a patent to the patent office, and if the patent office is feeling in a capricious mood, they may very well awarded to you. Has happened If Lauren hands me a piece of her fan fix and puts it on my desk and I swipe it off into the trash, Am I going to

get sued? No? But if Lauren writes a piece of fan fiction and electronic format, and you design a program specifically designed to allow you to read fan fiction, and that program includes the swipe to remove command that you have programmed yourself. You have probably uh interfered with that you would need to pay a license. It's it's chrono trigger fan fiction. By the way, I'm glad we I'm glad we got to the bottom of it. Thanks Joe.

That's a relatively okay, thanks for fan fiction about I think. Okay, let's tell the story of Minority Report. Where did all these ideas come from? Well, that's the thing is that it's not like they just invented the ideas out of whole cloth. That what happened was Steven Spielberg who really wanted to have a vision of the future that would

be believable. Yeah, something where you feel, yeah, based upon the way the world is right now, I could see the kind of technology existing, and I believe the Year of the movies somewheround. Yeah. Yeah, I think originally it was meant to be got pushed back right, right, so push forward? Well, I guess it all depends upon your point of view. But yeah, it's it's um at either

at any rate. What's interesting is not just that the technologies have started to appear, but they have appeared much more quickly than you would anticipate, given the fact that the movie is based in twenty fifty, we would we would now think that the technologies of Minority Report are probably a little too conservative based upon where we are right now with a lot of those technologies, even if they're in their infancy, by twenty fifty, they're going to

be much further along well ye years sorry, yeah, singularity. I mean, the fact this movie exists is even just it's it's just a pure fantasy. But at any rate, the he what he wanted to do was he wanted to say, all right, what's the Steven Spielberg. Uh. What he wanted to do was get the some some really smart people who are working in various fields trying to develop cutting edge technology. We're talking about stuff that's in the concept stage, not even necessarily practicable, or if it were,

it was something that was just a prototype. He wanted to get them together and talk about what the future might look like, assuming that these concepts and prototypes bear fruit, that they actually do become a viable means of doing whatever it happens to be. So the big surprises, he said, Now, let's not just have some writers dream up some stuff. Let's talk to actual technologists and scientists and see what they have. Yeah. Yeah, and so in he held a meeting.

He had a two day meeting with several people, including the folks who were involved in the movie, as well as experts in various fields. We're talking everything from computer scientists, automotive designers, architects, all sorts of different disciplines to kind of come together and say, all right, what will the city of eighty or the car of twenty eight or the computer of ty what will it look like, how will it perform? How will it be different from the

way it is today? Yeah, And they all basically just hold up in a in a hotel in Santa Monica. Yeah. One of the I read one of the participants said, I think it was Jarren Lanier said that they went under the guise of being a dental technicians conference. Yeah, something as boring as they possibly could manage so that no one would bug them. And yeah. The guest list included Jarren Lanier, Neil Gershenfeld, who was head of m I t S Center for Bits and Atoms, which is

the best name for an organization ever. Uh, the automotive designer Harold Belker, Sean Jones, who worked in DARPA's Unconventional counter Measures laboratory. That sounds like a good lab. Yeah, John under Koffler, who was the He was a member of the m i T Media Lab at the time.

There are actually quite a few video of the m I T Media Lab showing off an interface that looks essentially like the one you see in the movie, including the gloves, using the gloves to be the the uh, the point of connection between the user and the computer uh, and a lot more besides that. So there were some really important people in the in the space of design and engineering and science and they all kind of just

sat and hashed out ideas. I love some of the stories from that, Like they came up with this concept for a car that had essentially just a seat and like you could sit down and just relax and then the car would take you everywhere you needed to go. And Spielbridge reaction was, well, where where are all the controls? And they said, well, it's all voice controled, you don't

you don't need it in controls. And then the spiel brig said, Okay, where the actors supposed to do when they're just sitting in a car, And they yeah, you gotta show them doing stuff. This is why I think we also see we mentioned this and in the episode two in Star Trek, you would think by the time we get to the world of Star Trek, everything would be Yeah, you wouldn't need buttons, you wouldn't. I like

that you're doing the flailing button pressing. Oh. I was gonna get around to saying it eventually, Yeah, yeah, no, no, no, it's all the actors in Star Trek are either pressing one single button very effectively, or ninety million buttons all at the same time. Go yeah, yeah. The whole thing being that if you don't have buttons to press, you're really just a bunch of people sitting around. So it's a Star Trek economy. You gotta come up with jobs somehow.

So it's a it's a cinematic it's a cinematic uh concession. Right, you have to say, all right, well, maybe in the future we'll have this wonderful technology where we won't have any kind of the interface will be invisible because it will be so integrated into the experience. But for the purposes of a movie, we need to make some concession

so that there's something interesting to watch. Either that or we just cut cars from the film entirely because there's no point in having someone just sit there doing nothing, so then you had uh to look specifically at a couple of these people. U. Jarn Lanier is someone that we may have mentioned in previous episodes. Yeah, definitely. There was one I know in December called Virtual Eventuality in which we spoke at length about him, So so go check that out if you want to know a lot

about him. But in brief, he is the guy who popularized the term virtual reality. He may have even coined it, but you know who knows at this point. It's a lot of people were working on it at the time. But he definitely was one of the pioneers in virtual reality technology way back even in the eighties, uh, And he was in that work trying to create systems where a physical person can experience a virtual environment in a

meaningful way. And often we see that with a big head mounted displays, things that are allowing you to get a full look at the world. So when you physically move around, your perspective changes within that virtual world. This is something we're really pretty familiar with today. Things like the Oculus Rift are taking advantage of that same idea. But it means that you have to figure out, all right, well, what's the interface? How do you interact with this virtual

world so that the interaction doesn't feel artificial or unnatural? Right, So like holding a game controller in your hands and then interacting in this world. That puts up a little bit of a barrier because it's not like the way you would interact within the actual real meat space. It's a thing that you have to learn very specifically how how to use and to fine tune how you use it, the same way that we all had to learn how

to type at this certain point. Right. So jarn Linear, we should say, didn't just work on overhyped and mega disappointing things like virtual reality. He also worked on very popular things. Okay, no, that hurt a little bit, but it's fair. Yeah, he actually would go on to work on things that had user interface says that we're really

familiar with. Now. You if you look at Minority Report and you see those gestures, it also looks a lot like what you might see someone who's using a Microsoft Connect. He actually worked on the Connect project, and we'll talk

a little bit more about that too. So he's been Although you could argue that Connect is not necessarily that popular either, because a lot of people prefer to either turn it off or they went out and bought the Xbox version that didn't come bundled with the Connect once that, once that became available. Um, but yeah, he's he's certainly

one of the instrumental people behind this. In fact, according to their memories, we read an article that talked about this meeting in but it was you know, the article was written several years after that meeting, I mean like a decade after or more. And uh, and so you have to remember that people are relying on their memory of the event. But he said he believes he brought a working pair of those gloves that had been they you know, they've been playing with those in in various

developmental labs for a while to show them off. He said, you know, this way I could actually show off what I was talking about instead of just having people try to imagine the concept. And that, in fact, is one of the things that made his way into the movie. Yeah, and I've actually seen a video of people using gloves like that in action. It was I think a two thousand ten TED talk with John under Koffler. Yeah, another

person who was on that list. This guy is the one who was the former member of the M I T Media lab Um and has worked in various companies, one of the big I would say, evangelists of this sort of user interface. Yeah, definitely, And he apparently was the person who really worked to create the the gesture language used by Tom Cruise's character in the film uh To to sort through different data and select things, and and was definitely working with Spielberg to make it look

like this kind of VERTU will Orchestra conducting experience. So yeah, he has a ted talk that you can watch where he goes through all of this. And again, if you have ever used any sort of touch screen or a connect or any other motion controlled UH interface that's kind of like that, it's gonna look very familiar to you. Okay, But as we mentioned earlier, Minority Report was actually very prescient in a lot of ways of real technology that came later. I mean, did it basically predict how we

use touch screens today? I think Minority Report, well, I think it was predictive, but I don't necessarily think that the people behind the user interfaces were would necessarily cite Minority Report as the inspiration because this work. As we said in I mean, the whole reason why there were these discussions in the first places, because there were people working in media labs who were already kind of designing

this sort of interface. So what if Minority Report had never happened, if the movie had never been made, I think we would still be in that gesture controlled world. It's just that we wouldn't have the cultural touchstone of Minority Report to refer back to, right, And it might have been until uh later films like for example, Iron Man or something like that, that we might have seen that sci fi vision of the clear screens and all that.

You know, it would have been up to whichever production designer on whichever film came up with the idea of, hey, if we put images and text right in front of the actor's face, it's a really easy way to make it visually interesting while they're really just typing on a computer. Right, yeah,

yeah yeah. And in fact, if you ever there's there's there are people out there designers out there for actual uh software and hardware who lament the existence of such science fiction films, particularly Minority Report, but also Iron Man is an excellent example where they'll be told by the business side, Hey can you make this look like such and such, And they'll say, all right, yeah, what works really well on screen may not necessarily translate into a

usable system in real space. But we'll talk about some criticisms too, But the certainly the the hand gestures, a lot of that language did in fact translate over into real uses. And maybe some of that wash was actually attributable to the the depiction in minority reports, So that whole uh swiping and pinching to zoom, that kind of stuff, those sort of things I think may very well have become popular because people saw that in the movie and

they responded positively. They thought, Wow, that's such a cool way to interact a lot of sense, let's do that. Yeah, yeah, what you know, whenever I have like if I imagine I have multiple screens in front of me and there's something on one and I wanted directly in front of my eyes, I love the idea of being able to just reach out and grab it and pull it exactly. So, So it definitely was one of those that I think a lot of us intuitively felt this would make using

computers so much easier for this specific use case. That's an important note to make. So um, you know, and like I said, you had people who worked on Minority Report or at least helped with this brainstorming session who went on to work in some of these interfaces. So like the connect would be the big one, you know, But so's it's a self fulfilling prophecy a little bit. Yeah. Yeah, It's kind of like if if I were writing a science fiction movie. Uh, you know, let's say that's it's

ten years ago or so. Maybe not that long, but it's several years ago. And one of my buddies happens to work in Google's Project X department, and they're working on the Google self driving car, which has not yet become public information. And I'm trying to think of what's gonna set my my science fiction world apart. And this is, you know, I haven't talked to the guys that you know, I think self driving cars are gonna be a big thing. I don't think people in the future gonna drive, and

I incorporate that into my story. Then the news breaks later that Google's working on this thing and that how far along they are, and then everyone's like, wow, you predicted something. And then Google went out and did it. That's that's the perception, right, that's what it looks like, when the real real story is that I was able to jump on something, uh, because I heard a friend of mine say, oh, I think such and such. Now.

Of course, one of the ways that we can definitely say that movies like Minority Report, and specifically Minority Report in particular, have influenced things at large is in the esthetics of technology and popular culture. Absolutely, And in fact, that was mentioned by the author of one of the one interesting article I think we all read about criticisms of U I and Minority Report. Are you talking about

Christian Brown? Yeah, he's an animator and he wrote a piece in the All a w L called how Minority Report trapped us in a world of bad interfaces. I had some interesting observations. Yeah, And this goes into that that element I was talking about earlier, where something that looks really good and compelling on screen may not necessarily

be the best experience in real life. And the fact that even the designers on Minority Report were at a certain point aware of that because like your like your story with a self driving car not having any buttons, you know, they knew that that's not what self driving cars are really going to look like in the future, but they needed to give the actors something to do, right.

So it's the same argument I hear about Peter Jackson saying that they needed to make certain adjustments to Laurer the rings in the Hobbit in order to make it

more cinematic. Now, in that case, they were wrong, and I totally understand, and my my bias might be shown, sure, but you know, but you can see practical evidence of this in the Well, I guess, I guess anecdotal note that not that many people are really using the connect because it's still a little bit Yeah, I think, I think some people just well, it's because I think in part that no one has come up with the killer application of the technology that has made it so compelling

that people want to use it. It's to me, it's the same in a way as a the Nintendo approach, where they took a very they took a very bold step with the Nintendo we and then with the Wi You, saying that instead of trying to compete against the other console companies by by pushing the graphics and and the processing power of their devices, they wanted to revisit and change the interface entirely and make the the actual gameplay experience very different. And it was one that initially worked

like Gangbusters. It sold a lot of consoles, but it didn't seem to have a lot of staying power, which does start to raise the question are gesture controls something of a gimmick that are? They are very interesting to look at, and they're very intriguing to us as people. But once we start getting actual and I you know, not hands on experience, I guess, but hands off experience, that it loses its luster. Yeah. Well, I've read some

pretty interesting criticisms of gesture based displays. Yeah. One of the things that Christian Brown pointed out in that terrific article was that although a lot of these UI developers are are referring to the gestures that we all use today is being very intuitive and natural. It's it's really he was saying that they're really learned the same way that we learned how to interact with any other piece

of technology. Right, Like just as using a mouse to manipulate something on a screen, that's that's a learned behavior. You know, with a lot of practice, it becomes second nature. But it is learned, just as these these like you were saying, Yeah, we don't naturally reach out and try to manipulate stuff that's not there and pretend like there is something there and then expect a result if we're sane.

So and that was right. His point was saying that he felt that we would need additional elements like haptic feedback to feel something. And then once you start to feel something like the presence of something, as if it's really there, then we start getting closer to the way we actually interact with our physical environments. Yeah, I think that's exactly right. Gesturing with blank invisible canvas as your as your object is not natural. I mean, it's just

inherently an unnatural activity. It's as learned as anything else. But once you're touching, sure, that's a baby could do it. You wouldn't have to be told. How Now, that's not the only criticisms against this interface, you, Joe, you have a very a very practical one. Right. Well, no, I I can't remember where I've heard this, but I've heard somebody make this point before. Uh, and so it didn't

originate with me. But the point is simply that you'd get exhausted I mean, it looks really cool watching Tom Cruise stand there, uh and fling things around. But the power of something like a mouse and a keyboard is that it allows a lot of specific virtual movement to be conveyed through very little actual movement. Supposedly, even the quite fit Tom Cruise got worn out on the set of Minority Reports, so in between takes he'd have to

sit down and rest. While he was shooting these scenes where he used the big computer, he was just getting exhausted doing all the arm motions. So I don't know. Maybe you could sell it as like a combination UI slash fitness device, but would people actually want that. I mean, for one thing, you all already mentioned what happened with the connect um, But remember when the Wii came out.

When it first came out the Nintendo, we and you saw commercials of people using the Wii remote to place warts games, and they looked active, They're standing up, moving around a lot. They looked exciting and futuristic. Then came the reality you can really play almost any game on the Wii by sitting on the couch making tiny flicks of the wrist. And which game playing posture did most we users select. Uh, mine was switching on the Xbox.

You know that you didn't notice that, And when people like very first got it, they'd stand up and move her out. Yeah, and then quickly realized that they could just slouch deeper and deeper into their couch and and make the tiniest gestures possible. Yeah. And so I think generally the path of least effort is usually what people actually want. Yeah. I think, uh, in certain cases, like a party game, then something like motion controls has a lot of appeal to it. Right. It's kind of like

going out to actually play tennis with people. Sure, Sure, maybe like for the standing desk crowd, like like people who are actively trying to be more active in their day to day work life. Yeah. And I think also there's this element that if I'm playing, if I'm jumping around like a lunatic in front of my my video game console and other people are with me, that's a party.

If I'm jumping around like a lunatic and I'm all alone in my house and I'm playing a game that's a little weird, it's entertaining for your neighbors, though it may also be entertaining for whomever has access to the camera on my connect. But that's going into more of a kind of dark like hacking conspiracy sort of thing. So no, I just imagine they've got like a streaming show called like Jonathan Jiggles, you know it's that would

be accurate. That'd be about the right name for it. Also, you'd have to have a much bigger office environment, wouldn't you. I mean, because otherwise I'd just be smacking Joe in the face right time I had to swipe something off the screen. And that's yeah, Yeah, it does really seem like this would not work in a place where people were looking at well, it doesn't really work for all

sorts of computer applications. That's the real thing we're getting down to is that if if you are playing a game, or if for some reason you need to be able to sort through a lot of images, for example, quickly, and you want to do it in a way that's visually entertaining for the people who are there, then this sort of approach kind of has it's kind an element of the dramatic to it, obviously, and so yeah, that I see the appeal there. But if we're like, like Lauren,

you're an editor, if you're editing an article. You can't imagine like having to gesture all the way through so that you can pan through an article and highlight a section that needs notes. I mean, that would be crazy.

I I already vastly dislike working with word documents, not necessarily microsoftware documents, but just just text documents on my smartphone or on anything that does not have a keyboard and mouse set up, because it's I mean, partially it's because I'm an old person and and and the touch screen interface is a little bit less intuitive for me than a keyboard and else I know better how to

get around something with a keyboard and mouse. But also just the functionality of touchscreen at this current moment, it's not such that it's caught up to that keyboard and mouse, right, And that's the that's the kind of we're transitioning a little bit, not just with the criticism of just your controls in general and the and the UI and Minority report, but also just the idea of what is the future of interfaces because the keyboard and mouse has been around

since the early Macintosh days. Before then, it was just keyboards unless you were at Xerox's Park facility but Apple with the Macintosh really popularized the mouse and keyboard combo, which then became the standard for home computers. Right, And it has remained that for decades now, and it works really well. It is a learned behavior, I mean, typing on keys and using a mouse. That's not a natural behavior either, but it's something that most people pick up

pretty quickly. It's a prizing to me that has remained such a strong user interface, such such a dominant one. I mean just the keyboard itself, of course, goes all the way back to the typewriter. So the fact that this has remained largely unchanged since it first debuted is really surprising considering the rest of the technology has advanced so far. So is the future of a user interface with a computer is it? Is it locked down to

that keyboard and mouse? Or are we going to see some other interface come into play and be the really new way of interacting with computers. Well, there's there's practical evidence of that as well, Right, Jonathan. You have a note in here about Xbox one sales as they relate to the connect. Yeah, So Xbox One when it originally came out, the connect was bundled with it right, and then at Microsoft announced that they were going to put

on a cheaper Xbox One going on sale. You know, there was already this this big story about how Xbox One sales were lagging behind the PS four, and everyone got excited did and it was revealed that the reason for the one hundred dollar discount was largely due to the fact that they would not bundle the Xbox One

Connect with that version of the console. So this answered two things that a lot of gamers were really uh hoping to hear, One that it would be cheaper, and two that they weren't going to be forced to have the connect because they just didn't see any value in that particular peripheral. And sales as a result doubled. So part of that is because it's cheaper. Part of that is because people were more excited about getting the console they wanted, as opposed to a console plus a whole

bunch of stuff they weren't necessarily excited about. Of course, one might be able to come back and say that people might be more interested in gesture based interfaces if they were just better and and yeah, I think I think there's two things you need to have a really

mature technology so that it's just like virtual environment. You need a really mature technology so that people are getting what they expect, you know, because the expectation of virtual reality back in the nineties was well beyond what it could actually deliver at the time. And then also on top of that, you need really compelling applications that take

advantage of it. Yeah, uh you know. Okay, So so addressing kind of the first half of this, I think that part of it is right now is that the sensors and software that are required to make light based just your control, like the connect work are are still really clunky and require these very carefully calibrated bits that are usually stationary, which limits the ways that you can use them pretty drastically. Um. That is starting to change with some upcoming devices, like, for example, that the Meo

arm band, which you guys might have heard about. It. It's a stretchy band that you wear over your upper forearm and it reads the electrical activity in your muscles to determine what gesture you're making with your hand, which is just so cool, I think. Um. It also has this nine axis inertial measurement unit sensor system, which is what we classically like macroscopically referred to as being gyroscopic sensors, but that's not really they're they're all they're all little

bits and bobs at any rate. Um. Yeah, those those will measure that the motion and rotations of your entire arm. So instead of it just being these broad gestures like these sweep your arm left or sweep your arm right, this now can be something much more precise. Yeah, I can tell when you're wiggling your pinky finger versus when you're wiggling your index finger in which in which way you're wiggling it. That's really smart. I hadn't even thought

about that. It's sort of like taking the mouse concept and applying it to your gestures. Oh sure. Yeah. And a lot of people seem to think that the MEO is going to change just your control and you know, of course, any any following technology that repeats it. Uh. The MIOS creators that Dalamic Labs have partnered with several smart glass companies, including Google, as well as software developers for everything from like construction work to bike couriers to healthcare. Yeah.

I can see this being a a good advance to the the approach. I don't know if again, I think it will be something that will be really useful for specific applications, and beyond that it may be more of

a curiosity. But I'm I remember playing a game on the Xbox three sixty Connect, so the previous generation of the Connect in which you were able to control a tank like a vehicle by you know, you'd put both arms forward to be moving forward, you pull one arm back to make a turn, you pull both arms back to reverse, that kind of thing. But they had to be really big, dramatic gestures, which meant they look like you're wrestling with somebody in front of you in order

to do this right. But but it's very it's also very similar to kind of the gestures you would see for someone who had to try and let's say they're presented with a digital image that seemingly has three dimensions.

You know, it's on a two D screen, but maybe it's a globe, for example, and in order to turn it, you might have to, uh, put your hand out a certain way and then slowly move it to the left or right, or you might have to put both hands out with like a foot apart in space and then move one hand towards you while you're moving the other

hand away to have it rotate. Something like this, you could have very precise controls where you just have two fingers out and you just very gently twist your wrist and that would be precise enough to be able to give that same sort of command that changes things dramatically when you have that precision. Yeah. I mean other companies are working on this sort of thing too, and for

all kinds of different industries. There's, for example, um, there was a kickstarter on these gloves that image and Heap the musician has famously used and did a ted talk for I believe I've mentioned it before on the show The gloves that uh that that similarly can control stuff

and uh. And there's a few companies that are designing infrared systems that would integrate with your car's dashboard and your smartphone to let you control your music, your A C, your maps, and and eventually possibly even like like monitor you to make sure you're not falling falling asleep at the wheel, stuff like that. That's useful. Yeah, what's it do if you start falling asleep with the wheel, you

get a shock? Blair horns. I'm not sure. I am told at least that sending electricity through the body of someone who's falling asleep while operating a vehicle may not actually be the best way of making sure they wake up. Just starts playing Take Me Home Tonight. Yeah, yeah, exactly, just it. It immediately goes to any eighties station with power ballads and things like that, and the windows roll down automatically, and yeah, the future so bright. I gotta

fall asleep at the wheel. But yeah, at any rate, this gesture control issue is not the only problem with these minority reports style. Yeah, we we mentioned another one earlier in the episode, right, Joe, Yeah, how about clear screens? How about clear screens? It's certainly not just minority report.

This is almost ubiquitous now. And you want to see looks high tech, right, Well, clear screen so you'll see there's a transparent glass pane somewhere and either displayed on the glass or hovering out in front of the glass because work. Yeah well, I mean this could maybe work, because you still got a screen. Basically, the beginning of this is is starting to show up. But yeah, yeah, well, the problem is you'll see it from camera angles where

it wouldn't actually show up in reality. Right, But you'll see data. You'll see data on clear glass or floating out in front. We're talking information, not necessarily the character start or you'll see visualization whatever it be. It might be. They might be watching kitten videos all right now, Obviously,

transparent start watch and kting videos. I'm sorry, please continue. Obviously, transparent displays, whether they're solid screens or the hovering virtual displays, have cinematic appeal because they not only look futuristic and kind of beautiful, but they make it easier to see both the user and the data or the visualization on the screen from many different angles. You can shoot from in front or behind or wherever you want, and you

can see it from all the angles. If you're looking at an actual opaque computer screen and you want to show birth both the person using it and what's on the screen at the same time, you don't have a lot of options, right, It's it's pretty much over the shoulder or some angle not too far from that. Yeah, are doing that ridiculous thing where they project the light from the screen. The character has to wear glasses so

that you can see what's actually on the screen. Yeah, So I don't think that clear screens are going to become a big thing in the consumer technologies. Please please explain why you feel that way, Joe, Well, one of the one of the reasons is that I would think screens like this would be very sensitive to light conditions in the room, right, the just the room where you're using it, much more so than the screens of today.

I mean, you can have a bad enough problem even with the highly optimized opaque screens we have with glare right, if there's the sun behind you, or you know, you're sitting next to a window. If you're looking at a clear screen, that just seems like it opens up even more potential problems for not being able to see exactly

what you're supposed to be able to look at. Yeah, if you don't have an opaque back and a source of light is on the other side of the screen, and that's going to interfere with anything you're looking at on that transparent screen. Yeah, So in that case, it seems like it would only be useful if it were a stationary transparent screen in a fixed place where the conditions behind it don't change, so like it's in front

of a wall or something, in which case, what's the point. Yeah, sure, sure, I mean, furthermore, how obnoxious what it be if you're trying to get work done and and you have a clear screen, so your coworkers behind you or I don't know, like doing a little dancer goofing off or whatever it is that they do. That's often what we do on my aisle. You guys are on the other aisle over My aisle is the Josh and Chuck aisle where Party USA every day. He's on the crazy aisle. Lauren and

I are on the boring eye. That's the way that we like it. I think that's fine with me. I'm an extrovert. I I did my aisle okay. So here's another problem with clear displays. As of now, they tend to force the viewer into very narrow viewing angles, so it's hard to see if you're looking obliquely. This wouldn't be that big a problem for one user if you're looking dead on, but it's harder to share screens that way. If you want to have somebody look next to you,

it gets harder and harder. It starts to be a personal space issue at that point, right, it might be difficult to discern the intended relative depth of objects in a field of vision when there's a transparent screen, or you could just have mistakes, right, you could have trouble telling what you're seeing through the screen versus what's on the screen. Uh. Another thing is systems like this today

tend to be very expensive and hard to produce. Uh. Though, just this year, one thing I did want to point out is that researchers said an m I T Lab announced they had been developing a new method for producing C through displays based on nanoparticles that react different colors of projected light, producing images on clear glass. So this new method would, according to them, be relatively cheap and simple,

and it would offer a wider viewing angle. In a demonstration, they used silver nanoparticles which produce these blue images, these blue visualizations on the screen, and they said that other particles could be used to attain a full color effect. And even eventually, they said, you might be able to turn this stuff into basically just a plastic film that you could spray over the top of any clear pane of glass. That would be amazing, So you could have

like smart windows that way. Yeah. Now I can see places where this might actually be useful. Say, if you want to have a heads up display on the windshield of a vehicle, right, okay, Or if you're talking about a device that's optimized for augmented reality, like uh, like smart glasses or a window of some kind. I don't know what purpose it would really serve on a window that wasn't on a vehicle, but maybe there's something I

haven't thought of. Well, I mean there's there's things like I've seen for smart windows, things like, uh, you know, up to date weather predictions things of that nature. Is just stuff that you're looking at the window, you're maybe you want to control, uh, the how dark the window is, because that would that could be incorporated with an LED kind of set up. Also just other stuff like you know,

like stock tickers. It tends to be all the sort of widgets that they tried to force on our televisions about seven or eight years ago, they're now trying to force on our windows. Yeah. Yeah, And so I can also see it if you've just got some kind of general device in the future that's for augmented reality. It's a tablet that you're supposed to hold up to the world and see through with data added or something like that.

But if you're talking about normal laptops, tablets, phones, things where you're just mainly going to want to be using the internet, checking email, composing documents. I really don't see the appeal of clear screens. I tell you one thing I would love to see in a clear screen uh

implementation in the movies. I think it would be both funny and and amazing at the same time, which is that if you're viewing it from the angle of the user, you get whatever the supposed whatever you're supposed to see, but if you go around the other side, you get the backside of that. So if it's a video of someone talking and you go around the back back of their head, yeah, I was like, that would be funny, it would be pointless. But because that's the source, that

wouldn't actually work. But I mean, unless you went to a lot of trouble, right, It's essentially the same idea as the freestanding hologram, except in this case it's on a screen. You know, the idea being that you can walk six degrees around the projected thing and see it from all angles, except in this case, because it's on a screen, you would only be able to see the front. And like if you're watching a YouTube video, you literally have to have a companion video filmed from the opposite angle,

which is why I think it'd be hilarious. You just you never bother explaining how this was ever achieved. It is. That's pretty good. Okay, I want to talk about another thing in Minority Report. Okay, voice controls. Yeah, this is something that we've seen a lot of work done over the decades. I mean, even a futurist rate Cartswile worked

quite a bit in voice controls. Yeah, but basically they were one of the first well not voice controls necessarily, but voice recognition was one of the first things that computer programmers like Kurtswile started working on way back in the day. Yeah. And uh, it's tricky stuff, right, I mean, you you have a lot of different variables to contend with, but in general, old we've seen a lot of progress.

So there's a lot of difference between simple voice control and voice recognition and speech control which implies natural language processing. Those are two totally different things, but we made a lot of progress in both, uh, and and so it's been really interesting to see that. However, even seeing that that progress, I'm curious this one's definitely more anecdotal for me because I don't have any surveys that back up anything.

But I'm curious how many people really regularly use voice control for their various devices, whether it's a smartphone or a tablet or the Xbox one Connect also has voice controls. Xbox turn off. Yes, now, if you are still listening to this podcast, you are not listening it to it on an Xbox, nor are you uh, or you have disabled the voiceover controls, and if you're not listening to you just turned it off. I don't personally know anyone who uses a lot of voice control options, at least

around me. Maybe it's one of those things that they feel kind of foolish doing, and so they only do it when when they're kind of hanging out by themselves. I thought people mostly only did it in the company of others because it was funny yeah or that thing. Or they might do it in the car, right they might. They might have if they have a setup where they can a Siri what is my armpit taste like? Or or or something along text my wife, I'll be home late,

the kind of thing. I do know one person who does use it um at least I've witnessed him use it a few times, U daily tech news show host Tom merritt I. So I hung out with him at dragon Con this past weekend, and he was using it in order to text people, to call people, so that way he could keep his eyes up so he wouldn't

run into folks. Uh and dry and Con is an extremely congested area for pedestrians, Lots and lots of people, so add into that the fact that half of them are looking down at a screen, and you definitely have some issues where you could be bumping into people, and you've got all these costumes and everything. So he was using it because it was a way for him to navigate without, you know, putting his his gaze down on a on a screen, and it really helped him out.

So I can understand that, But personally my own use, I get very um self conscious about not not so much talking to a device while people are watching. I mean there's a little bit of that too, but just how accurate is it recording what I'm saying. So I've used voice to text in order to send text messages, but then I go and I prove the message before I send it, so I make sure it's not saying something just totally ridiculous, um, I mean beyond what I

tend to text people and unintentionally ridiculous. Right, And at that point, I've spent enough time initiating it, speaking into the device, reading over it, and then making any changes. I could have just typed it in that time. So maybe voice controls still are not quite at the level of sophistication where that it feels really natural. But again we run into a lot of the same issues we see with gesture control, the idea that in certain environments

voice control just isn't ideal. In our situation in our office where we have writers and editors, clearly this would not be a great a great user interface because we would all be interfering with one another, not to mention driving each other up the wall with all the talking. So it's another one of those that I I love the idea, but I don't see it replacing, at least for the actual home computer experience the mouse and keyboard. And keep in mind, voice controls for PCs have been

around for years. They've just recently gotten good enough for you to actually use them, but they've been around for a while well at any rate, if we don't live in a world that is dominated by the user interfaces of Minority Report, I'm sure we're still going to see examples where they will come into play. Yeah, I do hope we get to see those little spiders that crawl into the building and look for your eyeballs, right, Yeah, those are my favorite future technology. You know, I've already

had surgery performed on my eyes once. I did you get one of those things peeling up your bandages trying to peek at your eyes? No, I didn't get one of those. Um, yeah it was. It was more octopus

like than spider like. But at any rate, if you guys out there have any suggestions for future episodes of Forward Thinking, Maybe there's another science fiction film that has a vision of the future and you really want us to discuss it and look into it and talk about whether or not it's realistic, or if it's not realistic, why let us know, Or maybe there's just some other topic about the future you're really curious about. Send us a message. You can drop us a line on Twitter, Facebook,

or Google Plus. Our handle at all three is FW thinking. We look forward to hearing from you, and you were here from us again really soon. For more on this topic in the future of technology, I'll visit forward thinking dot Com, brought to you by Toyota. Let's Go Places,