What is the future of robotic hands?

mostly replacing us. Uh yeah, we sometimes have to take stock of the things that still fall under the category of humans can definitely do them better than machines. Right until recently, one of those things was playing the game Go not anymore anymore joined chess and checkers in the field tik tac toe. Yeah, the computers are so totally better than the hermon Nuclear war Yeah probably. Yeah, Well, I mean in the movie War Games, it's it's definitive.

You know, when I said tick tek to, I think what I meant was connect for now that is a solved game that computers can always beat us in. But I'm pretty sure they can at least force a draw or tecto. But anyway, Yeah, so always doing better than us. Of course, they can drive cars better than us. That's been clearly demonstrated now with one crash out of one crash caused by a robotic car out of all the ones that have actually all the miles and miles millions

of miles driven by robotic cars. I think even that one's kind of question. Well, you're talking about where it went, where it got hit by the bus. I was trying to merge Google. It's Googled themselves said, you know that they accepted responsibility for that, but partial at fall it was a little bit of a backheaded acceptance because they said, like, well, we just anticipated that the bus driver would slow down there, Like well if the bus had been not a shirt, right, Yeah,

so there's that. You know. One thing I remember coming across a couple of years ago, I think is that even now now, computers are even better at reading micro expressions on human faces than humans are. Yeah, so you know, they can tell when someone's disgusted or bored or whatever better than we can. Yeah. I definitely I have encountered humans that I know clearly lacked some of that ability where I'm like, I can't get out of this conversation. No matter what I do, you have to actually tell

them explicitly stopped talking to me. Yeah, I've also been guilty of doing that. I just carry around plasmic grenades and what I'm done with a conversation and just stick it right to their face. I was just going with the ninja smoke pomps. Just then run. You know, for a while the robots were pretty bad at walking on uneven surfaces, but they're even closed in the gap on

that those Boston Dynamic spots. Although if you watch that DARPA Grand Challenge where it had the sequence of tasks that the robots had to be able to complete, we saw that robots still find certain tasks really challenging, especially bipedales, like like opening a door and walking through it. Well, bipedal is a terrible system for anyone to use. Even our own spines are really meant to be like Crawley spines. No, they're meant to sit in front of a computer. I

sure hope so, because that's kind of what I'm doing. Well, it's true. What one of the biggest problems left for machines is dexterity diverse object manipulation on the small scale, So you deal with this skill every day. You use it all the time handling various objects, especially delicate objects with precision. One definition that comes from some researchers that we're going to cite later in the episode puts dexterity as quote many independently controlled degrees of freedom, speed, strength,

and compliance. I think that pretty well summarizes it. It's what you do with your hands to get all the different jobs you need to do done without crushing everything you touch in the pro or injuring yourself. Yeah. For example, Uh, we can use our hands to manipulate all sorts of different objects pretty readily, even if we haven't come into contact with them before. Just through the experimentation of touching the object and then you get a feel for what the object is like, you kind of I get an

idea immediately. How much pressure should you use when picking up the object. When it's something squishy like a banana, you know not to squeeze so hard that it squirts out both ends. But also if it's like a prickly, uh sweet gumball or something, you realize those points, like if you use too much pressure, you're going to hurt yourself and you know to be gentle with it. And you also get a feel immediately how heavy that object is, so you know how much force to use when lifting.

These are things that are intuitive to us, but robots don't have intuition, except they kind of can if you go with machine learning. But we'll get to that. Yes, I think that as well, said Jonathan. So think about it this way. When you pick up a banana and then peel that banana, and then pick up a knife with your other hand, and then slice that banana over a bowl of cereal. This may be a simple action to you, but this is horrifying warlock magic to robots. Right.

We we have nothing even close to being able to do this unless there's a robot that has been painstakingly explicitly programmed specifically for this task, which I haven't heard of. If there is, and if there is, it makes you wonder what better things people could have done with their time rather than training a robot to slice a banana into cereals robot. But yes, that's true. That's that's a kind of good example of the the idea of explicit

programming versus a sort of generalist adaptable programming. But yet gentle variable, dextrous and deft control of objects in the hands. That's just something that robots are not at yet. And another thing is that watching video of robots trying to achieve this is often really funny. Now. I watched one recently where they weren't very um advanced robotic hands. They were actually pretty simple clamps. Clamps there, clamps. Yeah, it was essentially it was a two pronged kind of hand, right,

like like just just clamp. Yeah, and uh, but it had a box full of different objects, and the robotic arms or the hands rather, we're quote unquote learning how to pick up those different objects. And they would attempt to pick up an object and start to learn from uh, trial and error how that was not working properly or what did work properly, and it would try to replicate

that in the future attempt. And it was a box full of tons of different stuff, like, uh, staplers, and it was mostly office equipment type stuff staplers and and and tape and that kind of thing. And they were supposed to pick up the staplers and so for a while, you know, they're just kind of fumbling around trying to find the best way to get a grip on the staplers and eventually started to figure it out pretty quickly.

Then they started to go with more advanced approaches, uh, which the hands were learning on their own, direct developing these techniques on their own where they would knock an object that they didn't want out of the way in order to be able to get a grip on the object they did want. So it's kind of like you know a human where you see a pile of stuff and the thing you want is under something else. You just kind of move the thing on top in order to get it to the thing that you actually want.

And it was fascinating to watch. It was very funny because at the beginning it was just chaos. Right, It's just robot hands going everywhere in this box of stuff. It's like the hands in that office simulator game talk about just throwing things randomly in different directions. Exactly. Yeah, So, okay, now we're we're criticizing the current dexterous nous, dexterousness, dexterity,

dexterity of robot hands. But you might be thinking, wait a second, now, I thought that there were industrial robot arms and so forth, all kinds of robotic effectors that carry out very precise jobs like machining, assembly, and manipulation. Job and they do, yeah, they certainly do. But these robots have been explicitly programmed for a very specific job

that they do over and over again. It's worth the time for somebody to figure out exactly what motions they need to make and teach them how to do that, because they're going to be doing the same thing maybe for the rest of their lives or at least for

one long production run. Right. It's it's the idea of of using a robot instead of a human for those those uh jobs that have really tedious, replicated work that you have to do adding a single part to a larger thing over and over and over, right, or welding a door to a car frame, that kind of stuff where you know it's something that would be dull or dangerous or dirty. That's the three DS that we often get for the kind of jobs that robots are best

suited for. And as long as it is a task that is going to be exactly the same time after time, you don't have to worry as much about the um the general dexterity. You just have to you just have to make it really good at specific movements. Yeah, one thing that it does. And so industrial robot might be able to, for example, place a ball bearing in a machine slot with precision over and over again. But if you try to get that same robot arm with its same end defect or to open up a ketchup packet

or something, well, let's see how well it does. End defector, by the way, is the industry term for thing at the end of a robot arm. Yeah, it's the part of the robot that the robot uses to interact with the world. So this could be like a welding tor to drill, a puncturing style, as a suction cup, a claw, a clamp, or maybe a hand, maybe a humanoid hand

with five fingers. But hey, you might also be thinking, now, didn't you guys talk in the past about robotic surgeons and how these robotic these surgical robots could give surgeons really incredible precision in laparoscopic procedures reaching into the body, And yeah, that is also true with precision cutting sutures and a tiny, tiny little movements. There is precision in

these robotic tools. But a couple of things. Number One, they're controlled by humans, So What they do is they translate large movements of a controller's hands into very tiny, precise movements in reality. And this is something that anyone who has used a computer in the last thirty years has had experience with, because your mouse does the same thing. When you move a mouse, you're moving it, uh a small Well, it's it's kind of the opposite side. That's

the same principle, but in opposite right in scale of me. Exactly, you move the mouse a little bit and the the arrow will move a greater amount on your screen. Because if you had to move the mouse exactly the same distance as the cursor needs to travel, you would need a really big desk. But yeah, so it's the it's the same principle, just in reverse, right. Uh. And also the robots in these cases, they have a limited number

of objects and tools they're designed to interact with. Uh. And as we said, it's directly controlled by human operator. So it's not the same thing as the needing a

robot that has the kind of dexterity of a human hand. Now, some robots can achieve more variable motion, uh, not by having a really really powerfully generalist in defector like a human hand, but just being able to like switch out tools, right, Oh sure, yeah, just sort of like a plug and play kind of option of like, well, I need a clamp for this, and I need a blowtorch for that. I'm not going to put a blowtorch on the clamp. I'm going to switch out my robot in defector hand.

So clearly it's like Dustin Hoffman's hook in the movie of the same name, where he could switch the hook out his other hook he has. He has one that is a double cigarette hold, right, and he has one that's a magnifying glass. Yeah. Does he have one that's a hand mirror? He might have. Yeah, he did have one that was a mirror. You're right. Yeah, it's when he's being um attended to by smee. Yeah, where his his mustache is being perfectly curled. I can I can

respect that. Yeah, yeah, but the same sort of thing. Well, anyway, so we've discussed all the reasons that robots generally just have a very specific type of end defectory that's for the job they do. But why would you maybe need a robot to have something more like a human hand. Joe, let me let me run this by you. We humans

we've been around. The stuff that we've built, we've built for ourselves for the most part, and and we humans, uh many of us, have hands, and so the stuff we have built has been meant for creatures that have hands. And therefore the world that we interact with, particularly the the man made portion of that world, is really hand centric. So if you want to interact with that world, hand

are pretty much where it's at. So what you're saying is that asking a robot to move around in the human world is unbearably cruel and frustrating if that robot does not have hands and you need it to do things that hands would come in really handy. Yes, I like, if you ask a rumba to open a door, I'm sorry, that's just not happening. Trying to get clamps to break

the yard for you, It's gonna have a hard time. Yeah, I mean what I Here's the funny thing is that as a left hander, UM, I encounter a tiny percentage of the frustration that if a robot could feel it, it's certainly would feel in a world that is made for people who are predominantly right hand dominant. Right, So I have this little experience where when I want to

interact with tools. I need to use certain things. Sometimes it is counterintuitive and difficult, and sometimes even in the case of chainsaws, dangerous for a left hander to wield. And so uh it there and think, well, I actually have hands. It's just that I'm dominant with my left hand,

not my right hand. For robots that don't have hands, then these problems are magnified by orders of magnitude, and so so the robots that exist today with very specific in defectors are not really a problem because they don't come into our world. They're not supposed to be able to do all the things we do. But people project that in the future, maybe in the next couple of decades, we're going to start introducing robots that are not the

specialists that robots have always been, but their generalists. And they're not segregated from us in the way robots have always been. They're integrated into our lives. They live with us, and they do the same kinds of jobs we do. And so, for exactly the reason you point out, these robots need to be able to touch the same things we touch, and touch them in the same way, manipulate them in the same way to to be able to handle the stuff we handle without smashing it or dropping

it or not knowing what to do with it. Yeah, otherwise you end up having to redesign all the tools, right, you'd have to redesign the tools for the robots, which seems like that would be a whole lot more work, depending on how tricky it is to really get a good set of robotic hands. Also, I want to point out I'm so sad that our listeners can't watch this podcast because we are just naturally gesturing with our hands

a whole bunch a lot to begin with. That's true. Yeah, I do make a lot of crab claw emotions in general, and most podcasts, that's true, full of clamps. Yeah. But another thing, so one of them. One of the things is that we want generalist robots rather than specialist robots to be able to integrate into our lives in the future. But the other one is just the The human hand is an amazingly remarkably powerful machine. It's it's a it's a miracle of nature. It can be adapted to so

many different jobs. And this isn't just because it's natural. I mean, I'm not saying any biological version of an indefector would be able to do this because think about a crabs claw or an ants four limbs. They can't do all the things that the primate hand can do. It's the primate hand, and specifically that's so amazingly powerful. It can push, pull, punch, pinch, slap, climb, throw, place things definitely, I mean it can do in thousands of things.

And and it can do all these things with objects of vastly varying size, weight, and shape. The minimum roboticists say that you need to to approximate to human hands gripping and manipulation capacity is nine degrees of freedom. That's that's three fingers capable of moving in a three dimensional space. Yeah, and I feel like I've seen robotic hands that are basically three fingers. Yeah. Yeah, it looks a little bit

like a like a nightcrawler hand. Yeah, um um. But but a fully functional hand is a lot more complicated than that. Yeah. The human the human hand has more than twenty independent degrees of freedom. I think I've seen somebody say twenty seven somewhere. That might be a disputable number, but definitely more than twenty. Uh. And it's controlled by more than thirty muscles, which in in robotics terms would be actuators. Yeah, and so, uh, you know, when you

think about something that complex. Now, let's let's be fair. The human hand has had the benefit of millions of years of evolution to lead to that that that approach, right. Robotic hands, on the other hand, have had a few decades. Right, So we are trying to replicate what took nature millions of years to create. And uh, and so we shouldn't be so not skeptical, but so dismissive of how quote unquote primitive robotic hands are right now, because honestly, they're

getting to be pretty sophisticated. And also, I did want to point out that, you know, just because the human hand works so for us, doesn't necessarily mean that it's the only design that we can come up with. It can do this kind of amazing stuff. There's some really cool generalist end defectors that are being designed that look

nothing like a human hand. I wanted to mention one sort of as an example of how advanced they can be yet yet still failed to match our hands design ultimately, and I think we've we've mentioned this type on on the show before. Um, it's based on jamming transition technology and okay, and in physics, jamming is what happens when granular materials that would usually flow, like a sand or sugar or coffee grounds or whatever, they become rigid when

they're like really packed in. Yeah. Like like if if, if you've ever handled a package of vacuum sealed coffee grounds, it's a it's a brick. It's solid, completely solid, but but if you open it up, you can pour the grounds into your coffee maker. Um sand flows through our glasses, but in a hack sec it becomes a solid, bouncy unit. Yeah, so, and I like that. We I think this is the second mention of hackey sex we've had in this episode so far. Good times. I think we actually mentioned hacky

sex before we start. It's okay because it's always in our hearts and sometimes on our knees and feet. Jonathan, were you a high school hacky sack? I am not answering that question, Joe, please go on, Lauren, no problem, Jonathan, Okay, So, So, jamming transition, which is the jamming transition technology, is the word I used before, the term I used before. A jamming transition occurs when grains of stuff become dense enough

to act as a solid. And this, as it turns out, is a really useful property because researchers figured out that if you fill a soft rubber ball with a granular material and then attached that rubber ball to the end of a vacuum tube, you can quickly and easily move air into and out of the ball, thus changing the density and the behavior year of the material inside um. And you can use such a device to to grip and handle objects of all kinds of different shapes and

sizes because they mold to it. And and then like if you've got a lego brick, the ball can bend around the top of the brick or the top of a bottle cap and solidify around it in order to pick it up. Then it can soften again to put it down. And this is pretty amazing. Yeah, yeah, you might have you might have seen one that the bright green versible um. It looks like a like a weird little tennis ball on the end of a tube. And and about a year or two ago, I think there

were a lot of videos going around interesting. But yeah, yeah, it's a it's great. It's really fascinating and beautiful to watch inaction. Also, it can't play a piano at all. It would suck at that. What's amazing to me is that this actually reminds me of I saw a comedy show. It was improvised comedy show, and one of the characters in the show was a robot that had become horrified that had had killed his best friend because did not realize that humans can't reboot, and so had replaced replaced

these hands. He would say that over and over in the episode. He replaced his hands with oven mits filled with cookie dough, is what he described it, so that way he could no longer hurt anything. It would just flop against people and uh and it was. It was hilarious and heartbreaking at the same time. So when you're describing this like, I can't believe that that incredibly weird improvised comedy thing I saw fifteen years ago is kind of sort of the same thing you're talking about. It's

just it's just it's a cookie dough. It would be some sort of granular type stuff. But that's amazing and weird. Are the are the future thinkers? I guess, I guess the Jules Verne of today. I have to I have to let him know. I know the guy who did it, so I have to tell him, Hey, you were prescient in your goofy makeup ups. Some people who have been working with robot hands even earlier than this improv team of yours. Um was it? Was it with Dad's Garage,

Christ's Garage. You guys, if you're ever in Atlanta, if you're from Atlanta, you should totally check out the work that Dad's Garage does. It's a terrific theater company anyway. Um okay, So, so even before Dad's Garage, people were working on robot hands. And if we were going to go into the full history of this, it would the history of the development in the technology. That would be a whole episode unto itself, probably a couple And maybe we should do that on tech stuff sometimes that might

be a great idea. Yeah, sure, since we've got some of the research right here, that would be a great foundation for a tech stuff episode. But I did want to mention a few of the particularly like key or or interesting stories from the very beginning days of the development of this kind of stuff. Cool. So, as far as I can tell, the first electrical mechanical five fingered hand is from nineteen six two. I bet that was

a horror. It wasn't a full robot it and it actually it was so beautifully designed that it wasn't such a horror as you would imagine. Yeah, it was intended to to be a prosthesis. And it's called the bell Grade Hand. It was designed by a biomedical engineer by the name of Rachael Tomovich and an electrical engineer by the name of Mio Drug at the University of Belgrade. He's the name the Belgrade Hand. And I'm really hope

that I'm not. I looked up the pronunciation of these names, but I don't know Serbian, so I might still be totally mangling them. It's way better than any time I ever say anyone's name in whatever language. So I think, I think you have acquitted yourself with honor. Excellent. Um, there's actually a lot of really beautiful images of their design because a couple of years back, and artist by

the name of another one, okay, Alexandra Demonovich. Yes, yes, she she built a digital model of the Belgrade Hand and then casted a whole bunch of copies for a gallery show she did called The Future was at her finger tips anyway, the Belgrade hand had five touch sensitive fingers that were controlled by a single motor, so that when any fingerpad came into contact with an object, all the fingers closed until the pressure on each pad was

was just about equal, thereby hypothetically gripping the object. Interesting, yeah, um, but not a robot. It was, you know, a reactionary machine piece. Um that. The first truly like programmable hand was a three finger device created in three in a collaboration by Stanford University and NASA's Jet Propulsion Laboratory, thus

called the Stanford JPL hand. Uh. The lead designer was this fascinating guy, Jay Kenneth Salisbury Jr. Who's a mechanical engineer who specializes in both computer science and surgical anatomy, which diverse interests. I know, I love it, um and and the Stanford JPL hand had three joints on each finger for nine total degrees of freedom with its again like kind of like nightcrawler hands, and and its motions were controlled by electric motors that were attached to tendons

within the fingers. But perhaps the most famous device from this general era was created in five. It was a four fingered hand from the University of Utah and m I T. That's called the Utah m I T Hand, and each finger on it had four joints for a total of sixteen degrees of freedom, and its motions were were pneumatically driven. Wait, four joints, that's more than real fingers have, right, Yeah, it is, isn't it to yea more? Yeah?

So it looks and only are we gesturing with her hands were referring to them staring at let me physically count the joints on my hand. Um. Usually it's only on the weekends when I'm just staring at my hands. But today I feel like we're really really making progress toward that. Uh and so. So the fingers were sort of like long and spidery. It actually reminds me of like a skeletal version of of like Mr Burns hands on The Simpsons, the three fingers and the thumb and

so hull um. The the kind of most advanced of the bunch was released in nineteen eighty eight or or debuted in because there were a couple of different models of it out of the University of Southern California and the University of Novi. Sad At Belgrade and that's called the Belgrade USC hand. It's based on the design of that original nineteen sixties Belgrade hand. Um So, it had a rotating thumb with two joints, and fingers with three joints each for a total of fifteen degrees of freedom.

And unlike these these previous examples, the joints were not independently controlled. They kind of took que from that Belgrade hand and also from actual human fingers because if you, I mean, you can independently move each finger, but all of the joints on each finger are kind of attached to each other, I mean, movement wise. So, um so, if if if a knuckle joint moved on this spell Grade USC hand, it would trigger the other joints on that finger to move as well. Right to sort of

curl in. Yeah, Like again we're looking at her hands and playing with them. Well you know, it's just try to just curl in your pinky. I can't do that. I can't do your other finger comes with it, not at all like that. That's a that's a like I'm point, I'm showing something to people who can't even see this. It's just Joe and Lauren to get Yeah, when I curl, When I curl in my pinky, my my ring finger is almost as curled in, and then my other two

fingers curling as well. For me, because I can't, Lauren is able to turn her pinky's in much more capably. Just I was able to just pull my pinky in after I forced it down with my other hand. That's cheating, cheating, but but there are some robot hands that can do that, where each finger can independently move with that full freedom.

This approach, however, was more like a human hand, and that you had those connections between the various fingers right right, So it's a little bit more naturalistic and probably energy savings as a result. Interesting. And um, and these these few that I've just mentioned, we're kind of the standards that that everyone referred to for for a number of years. Um, it wasn't really until the late nineties that a whole

slew of research projects involving robotic hands started going into development. Again, you know, the tech was kind of booming at that time, and also the the size and cost of computers was really starting to actively shrink, right, yeah, right, uh yeah, And so of course research has continued. I know one private company that has been a big name in in robotic hands or at least five fingered robotic hands in the past decade or so has been the Shadow Robot

company out of London. Doesn't sound terrifying at all, right, but they've yeah, they've got this product called the Shadow Dexterous Hand. And when I when I was looking at a research that was ongoing at like university labs with the robotic hands, a lot of them seemed to be basing their models off of the Shadow Hand, like making modifications to it and stuff. Right, the Shadow Hand itself is commercially available, so you can buy one. They do

not say how much it costs. I wonder why not you send an inquiry and find out, uh general rule of thumb if you have to ask, it's too much for you to pay. But but but it is. It is a really fascinating piece of technology. I know a shadow now. I don't know exactly how much their product came through in the final version, but they were involved in the creation of the Moley Chef robot right right, which is the one about those disembodied hands to make

the crab bisk that Lauren was talking about. Earlier. Yea um, so it's got hands reaching around the stoves, you know, adding ingredients and stirring stuff. And we know that the Shadow Robot company had something to do with contributing technolo lergy to that, right, So the hand that they make, the stexterous hand, has twenty degrees of freedom, so similar to a human hand. Twenty four joints total, and the

fingers and thumb are all capable of motion. And it can also flex its palm similar to what our hands do near the little finger when we close our fists. So when you close your fist, you see that there's a little bit of a fold right below your your pinky that this hand also does that. So it's not just an independent joint where the pinky just folds down over the palm of the hand. The palm itself, palm doesn't move at all exactly. So it has a hundred

twenty nine sensors, so pretty advanced, right. It provides telemetry data and it makes the hand useful for developing manipulation control systems. So you might end up getting a hand like this if you are actually working on the back end uh software side of things to control the hardware Each joint also has absolute position sensing, and each actuator has force sensing. That way, you can get really specific with the way the robotic can handles whatever objects you're

planning on making it handle. The fingertips have other sensors in them as well that can sense tactile uh sensations in other words, rough versus smooth, that kind of thing. Also temperature, so it knows if something is hot or cold. Uh. I mean again, these are things that we take for granted as human beings, but robots don't just automatically know that.

If a robot hasn't been programmed to sense temperature and react to it in some way, if you had a robotic hand and it came into contact with something too hot for the robotic hand to stand up to for any length of time, it would just get destroyed. It would be melted or burn. So uh, it's a very clever thing that they added. Also, it can detect current

and voltage um so you can have it. Yeah, you know, you're working with electronic systems and being able to detect, you know, whether or not current is flowing, something that you don't mind a robot doing, but you probably mind if a person is doing it. If you're a decent person, you mind if a person is doing it, because you're depending on the amount of voltage and current, you might have some pretty big safety issues on your hands. Um

And so really a cool piece of technology. And uh, you know, Joe, you wrote the script for the video episode that we did about robotic hands, which and I don't know when this podcast goes live. I have no idea if that video will have published yet or not, because I don't know what the publication scheduled for our videos are, so you might be getting a preview right now. But it was really neat to look into this and see the design and see how it moves, um and

I was really impressed by it. One other thing I want to add is you might think, all right, so what are these robotic hands for. I mean, they're not attached to a robot that's disembodied hand. So some of it is tell operation that idea of being able to control a robotic hand with a human controller to operate something that you might not want to be in the area for, like you're handling toxic material reels, for example. But another one is you could actually use this to

outfit a robot with these robotic hands. So if your robot has big old hooks at the end of its arms and you're like, finally the hands have come in, you can you can rip the hooks out, dump the hands on. Congratulations clamps. You're getting some fingers, right, Yeah. So the shadow hand, like we said, has been involved in like it's been used as a model for some

university research. And one university that I have seen plenty of interesting robotic hand research associated with is the University of Washington UH and a couple of researchers who have been associated with the University of Washington. Earlier this year, there was a cool piece in the I Triple A Spectrum about them creating this bio memetic hand that was trying to take the five fingered hand approach to make it more like the skeletal and muscular structure of a

real human hand. And they went to great lengths to make sure that they could do that is super cool. The way they did also, depending on your point of view,

maybe a little creepy. So so you know, our our normal robotic hands, which is a weird sentence the way to start a sentence, but but your traditional robotic hand it's it's trying to mimic what a human hand can do, and might be somewhat based upon the structure of a human hand, but we have to take a lot of liberties, right, like using hinges or gimbals in order to replicate a joint, for example, and it's not gonna have bones like your hand, right, And also might mean that the the motion that you

get out of such a hand is not truly representative of what a human hand does, or what it can do or what it cannot do. And that becomes important when you want a robot to be able to interact with our human world, right, you want the robot to be able to do essentially the same sort of things a human can do and avoid the stuff that humans can't do because it's irrelevant within the hum in the world unless it's something that ends up being useful. Right.

So they decided that instead of just using these hinges and gimbals, they wanted to take a different approach. So they took a dead person's hand, a cadaver, a skeleton from a cadaver, actually a skeletal hand, and then they scanned it with a laser scanner. So they got a really accurate scan of the bones in a human hand, and then they use the three D printer to replicate that by printing out artificial bones. Now, this meant that the joints where the finger the finger segments all meet together,

they are exactly the way human joints are. So it doesn't have the mechanical element of like a hinge or a gimbal, like a screw or something like that. You actually need ligaments, yeah, because it's otherwise it's just a ball and socket correct, right yeah yeah, just if yeah, I'll just fall apart. It won't won't do anything. Um,

so yeah, they use artificial ligaments. Uh. They also talked about it was particularly full for the thumbs because the thumb joints are are a little more advanced in a way you can you could say, then the finger joints are the base of the thumb. You can do a lot more motion than just bending it down. Again, we're looking at our hands and playing with them. This is great. I can't wait till we do the episode on feet and then we're all like this was a terrible idea.

So uh, they nobody cares. Just wait, we're gonna get a request now. So the muscles, though the muscles in the hand are simulated by ten dyna mixel servos and so they're using uh, mechanical servos to pull uh, the the cables that are replicating the muscles tendons. Yeah, and that controls whether the fingers are open or closed and the various motions of the hand. Uh. And they actually route the cable in a way that is similar to the carpal tunnel of your your wrist and hands, so

that maybe this robotic can could get carbal tunnel syndrome. Well, you know if it, if it actually worked long enough, then you would imagine that those cables would eventually wear out and so you would have to do some surgery on it. I doubt it would have to wear a brace afterward, but you might uns it to feel pain, right yeah, which I mean, come on, that's got to

be on the list. But that's a funny thing to point out because it does highlights another possible use for an extremely accurate replication of a human hand with a robot, which would be hands used for medical research. Yes, like if you wanted to use a robotic hand in some kind of futuristic setting, maybe as a scaffolding for regrowing regenerative tissue, or as a or as a model for replacing a human hand like Luke Skywalker and Empire strikes

by giving somebody a fully functional robotic hand. Now, we we have some hand prostheses today that are sort of referred to as robotic hands, but they're not quite as functional as a real hand would be, right, And and they in fact have said that that was their intent to create a platform that could be used in such a way. Also, you can control this hand using a Waldo. And I just wanted to bring that in because it's one of those terms that you've probably heard if you've

ever worked with puppetry. Uh, the puppeteers who are using a digital electronic puppets often use a Waldo to control them. Essentially, that's a remote manipulator. It's named after a character from a short story from a Robert d. Heinland piece, Waldo, who the guy named Waldo creates a device that does this thing. So when people finally did create a device that does this thing, they named it after the fictional one.

And so it allows for Tela operation. So in puppetry, this would allow you to do something like move Hoggle's mouth in labyrinth, so that the the actor inside the hoggle suit isn't also having to manipulate some other control to make the mouth move that's actually being controlled by

a puppeteer off screen. In this case, you would be wearing a essentially like a glow with sensors on it, so when you started moving your fingers, it would detect the motions and then translate that into commands that the robotic hand would follow, and it would mimic the motions that you make with your hand with just a slight delay. So it's a little it's a little eerie to watch,

but also really cool. Yeah, So we mentioned that that research was associated with the University of Washington, but there's

some more that we want to point to. Uh. For example, there was one group of researchers who in May of sixteen, they're they're presenting a conference present I think the presentation hasn't actually happened yet on the day we're recording this, but the paper has been released online presenting a conference paper called Optimal Control with Learned Local Models Application to Dexterous Manipulation. And this is research about machine learning for

programming robotic hands. Now they're working off of a sort of modified shadow hand model that they came up with actually detailed this in an earlier conference present station paper. So in the earlier paper, they bought a shadow hand robotic hand to use for testing advanced control schemes that they were trying to come up with with in their robotic hand movement laboratory laboratory. But they did some testing and they found that the actuation wasn't fast enough for

the dexterous object manipulation they were going for. So in this first paper they described essentially a pneumatic actuation system to improve the design to make it faster, and they emphasized that their goal was not to build new hardware for its own sake, but sort of in furtherance of designing newer, better control schemes for robotic hands. Uh so,

what's the deal with the control schemes. Well, this team of engineers and computer scientists from University of Washington, a couple of a couple of the same as that previous paper. They designed a system for robotic manipulation that can learn tricks of dexterity from experience. And that was that paper title I mentioned, So that the lead author, Vikesh Kumar.

He's a doctoral student at the University of Washington studying computer science and engineering uh and he he explained the context for his paper by saying, quote, hand manipulation is one of the hardest problems that roboticists have to solve. A lot of robots today have pretty capable arms, but the hand is as simple as a suction cup or maybe a claw or a gripper, so they're on our frequency clause grippers, yea, not clamps, but there may be wisdom and a clamp, but it can't twirl a baton.

Uh Now. So the goal was to advance a control scheme to create dextrous movements for a pneumatically actuated so that's pneumatic movements, you know, air air pressure. Uh. Tendon driven twenty four degree of freedom robotic hand. So this is building off hardware ideas that they established in the previous research. And this this thing would have forty tendons, twenty four joints, and more than a hundred and thirty censers.

Because even if you have a great robotic hand with vast nimble hardware and feedback sensors and all that stuff, how does it know what to do? Right? Think about the way you you reach out with your hand and pick up a grape and then toss the grape into your mouth. If you can do that, can you guys do that? I would probably hit myself in the eyeball. I practice sometimes, and it's it's harder than it looks. They make it look easy in the cartoons. But anyway,

it's not just one motion there. There is a series of probably hundreds of individual muscle contractions and micromotions to do this one task that seems really fluid and simple at the micro at the macro level. But imagine if you're programming a robot hand to do the same thing. Would you literally want to have to figure out how to explicitly program what every single muscle equivalent actuator has to do at every moment in this action. I wouldn't. Yeah.

So one of the authors on the paper, the senior author and lab director Imo Todorov. He was quoted in a University of Washington press release explaining this. So he says, quote, usually people look at motion and try to determine what exactly needs to happen. The pinky needs to move that way. So we'll put some rules in and try it and if something doesn't work, oh, the middle finger moved too

much and the pin tilted, will try another rule. So he compares many robotic manipulation systems essentially to an animated film. And I like this comparison because it looks great, But that's because it's been meticulously specified by programmers in advance. And this system, on the other hand, the one they're coming up with and describing in this new paper, is more independent, which is important if you want a generalist

robot that can interact within a human environment. Obviously you don't want to have to program every single conceivable interaction this robot might have, because you know you're never going to be able to do that. Yeah, you can't. It's it's like that would be an exhausting choosier and adventure. It could. It could be one where you send the robot out and you're like, guys, we did great work today. Did anyone remember what to tell the robot to do if it ever picks up a knife? Get it back? Yeah?

I mean it seems to me that if we want to have good generalist robots with like a humanoid range of hand motions, we're gonna have to go to the machine learning round. I have a hard time imagining how all of the motions of such a machine could be explicitly programmed. That seems to me almost like trying to imagine a natural language chatbot that has all of its conversation explicitly programmed. It's just ridiculous. You couldn't do it. Uh, It's probably not quite on that level, but seems close.

But anyway, So what did they do well? The University of Washington team has a computer simulation of the hand that can analyze its movements in real time and this can help teach the robot hand how to improve its motions through machine learning. That their term is reinforcement learning or r L UH and in the paper they describe their r L algorithm. It is vastly complex and I don't understand a bit of it, but it's online if

you want to go check it out yourself. But they used this optimized control strategies for the hand, uh and to use it to reach target poses and to manipulate a cylindrical object object. And Jonathan, what is that cylindrical object? It's a plastic two filled with coffee beans. Not sure why they picked full of coffee beans. But I like it. They're probably on hand to speak. I don't think they just I didn't find anywhere in the paper where they

described why coffee beans. Unless I missed it, I can't put my finger on it. Yeah, but anyway, apparently the very first um jamming transition ball was also a balloon filled with ground coffee. Wow. I think it's just scientists like coffee. Oh yeah, they just have a lot on me.

There's got to be a study on it. But anyway, there's a video demonstration they put online where they gave the robot a task and what it is is twirling in elongated tube of coffee beans in its hand, and it's using machine learning algorithms to work in steps to slowly teach itself how to rotate this object in its palm. So it sort of starts with it laying across the palm and it's trying to move its fingers to twist it out like a baton right that it's handing off

to you. And I watched this video, and I gotta be honest with you, when it got to the point where it reached the success rate, I wasn't entirely sure. At what point did we pass the fail. But I will say that it makes sense that this is a complicated because you think about it's a tube that's filled with it's not packed. It's not completely packed with so stuff from shifts to the weight shifts exactly. Yeah, so it is trickier than what it sounds like upon when

you first hear it. Right. Well, I mean it seems easy for you because it's intuitive because you've already done that, not machine learning, but your equivalent, the bio learning throughout throughout your early years, spinning how to manipulate object. I've been twirling tubes of coffee beans for decades. But if you watch a toddler try to manipulate object, it's hilarious with their hands. Yeah, yeah, they'll drop stuff and they

don't know how to do it right. Watching babies trying to feed themselves, they can't operate the objects yet, the fork or anything. Like. I feel the same way every time I try to eat a salad because like, the leaves are all different sizes and I just I can't. I cannot put them in my face. Yeah, it's I'm terrible at human ng. Yeah, I got it. Um, I understand entirely. You must be getting bad salads. I they're great salads. They're just they're just randomly sized just so

you get that wedge salad. You just look at your like, my arch nemesis, we meet again at any rate. I I do find it really fascinating that they're focusing on pneumatics for for this research, as opposed to the electrical motors as actuators, which which I think have been more so the standard, like the historically speaking, it was an outlier, a couple of outliers that were neeumatic, and so's it's

cool that that technology is coming around again. Yeah. If you watch these hands go through all their degrees of motion, like doing all the different joint movements as fast as they can, it's pretty funny. It's like, uh, just fingers slamming back and forth, and yeah, you hear a lot

of clacking. Yeah, well, wouldn't that be a great scene in a horror movie like this, So there's a there's a mad robot with humanoid hands coming for you, you just hear the clacking fingers clacking up and down, or or another great scene you see, uh, what you had presumed to be a human character this whole time, just lift up their hand and then go through that sequence super fast and you're like, all right, that's not a person.

No human can do that. So one last thing about this method that they came up with in the hand that they're using is that it's expensive at this point. So it's not cost effective for their method to be uh, sort of exported for industrial use or something. Right now, it's a research model, So what they're saying is it's

to improve core robotics principles. And so what they're trying to do is come up with control schemes, systems for designing control four hands of this kind, and hopefully they're researching conjunction with other types of research along these lines going on around the around the world, can help sort of build up a robotic control scheme consensus that will allow us to have better, smarter control of objects even

with the good hardware we have today. Yeah, so not even with that applying to the good hardware we have today. So we can think of it as like primary research in that field as opposed to heading toward a finished product, which it's still really interesting and uh, and we're in the early days. I'm excited to see where that goes. I mean, think about some of the applications we could

put a really advanced robotic hand toward. Right now, there's the obvious one of robotic prosthetics, which we've seen some of, like we've got the Luke arm for example. We've seen some some pretty impressive improvements in that space, but we're still lagging well behind what what humans have evolved into over those millions of years. But that's clearly one use case for robotic hands as we get them more uh well, as they become more similar to the ones that we have.

But there are other ones we could use too. I mean, like you you mentioned the using it actually not to replace a hand, but perhaps as a platform upon which you could study and research and develop regenerative medical procedures. Yeah. Sure, regenerative tissues. I mean that's going to be I think

a burgeoning field in future years. Yeah, I think it'll be really interesting to see, uh where this goes, right, whether or not we do see more generalist robots enter into the scene, because, as we pointed out, the hands are just one aspect of how tricky it is to design a robot that is capable of moving through the human world as a depth. As we do well let's be clear, there is absolutely not even close to such a thing as a good generalist robot today. It doesn't exist.

So it may very well be that within our lifetimes we don't really see a strong generalist robot. It maybe that we see a lot of specialized robots like the Roomba, things that were designed to to do specific jobs and nothing else. And uh, it may be that that's just the status quo for several decades. Then again, I wonder if you might have a specialist robot that may still

need humanoid hands. For example, if let's say we we fully commit to the chef robot like Moley, you know, with hands doing stuff all over the kitchen, and we say, now, it's not just adding ingredients and stirring, it's also doing all the chopping and the peeling and the washing, and it's got to be able to handle all our foods,

handle all the kitchen utensils. In that case, I would think, well, maybe, even though this is to some degree a specialist robot, it needs humanoid hands, right, Or if you were going to make a robot to change baby stipers, sure, I guess so. I like that. I like that that's the example you went with as opposed to something I'm seeing in our notes right now for the first time robuts designed for choking and strangling. Joe, I just put that

in there for a laugh. Okay, that's fair, all right, but yeah, we we we definitely think that the the work, the advancement. That's fair. That's fair. So if you have any suggestions for future episodes of forward thinking, you can email Clamps at how stuff works dot com. We will not get that email, but you can do that, or you could actually send it to f W thinking at how stuff works dot com and we will get that one. Or you can leave us a message on Twitter or Facebook.

We are FW thinking on Twitter. Just search fw thinking on Facebook. Our profile will pop up. You can leave us a message there and we will talk to you again really soon. For more on this topic in the future of technology, visit forward thinking dot com problem brought to you by Toyota. Let's go places,