TechStuff Classic: DARPA Robotics Challenge Review

It is titled DARPA Robotics Challenge Review. It's about the DARPA Robotics Challenge where they were simulating UH an emergency situation and putting a robot through paces of doing certain basic tasks in an effort to see if the robots could do them, you know, more or less autonomously. Turns out that some things that you might think of as being incredibly easy to accomplish turned out to be pretty hard. Let's listen in So, to start off, we got to

talk about DARPA itself. Remember that's the research and development arm of the Department of Defenses technically the US Defense Advanced Research Projects Agency UH and DARPA has been responsible for a lot of cutting edge research, largely in the mode of military use. But we have seen the benefits of that research hit us in other ways. Besides, you know, the military applications, So for example, our pa net, the predecessor to the Internet, was the platform upon which the

technologies that allow the Internet to work were developed. And that was a DARPA project that was back when they were first just called ARPA rather than DARPA. But we wouldn't have the Internet, at least not the way we have it today if it were not for this agency. So the agency has done a lot of stuff that has benefited us in many ways. I'll talk about another one a little bit later on in the podcast, kind of draw parallels to what happened with the robotics challenge.

So here we have this agency. They're all about research and development. They don't actually have lots of labs with scientists working in them. Know what this agency does is it sends out proposals to other groups and asks them to help work on various projects. Sometimes they hold challenges. In these challenges, it's kind of an open invitation for any group that can participate to uh to compete in some way, and the winner usually gets some sort of

cash prize. And so often the cash prize will be less than the investment needed to actually participate in the competition to build out the thing that you need to make for us to to win, but it's still a very important UH and prestigious event to be part of

if you are capable of competing. Now we're talking mostly about research departments engineering departments in universities, but there are also some private institutions, not just university type things, but actual research and development labs that will participate in DARPA projects as well. And it's collectively that these teams were kind of they're the ones that are pushing the technology forward,

and DARK is kind of the facilitator. They're offering up the possibility for people to actually UH participate, and not only the possibility, but the goals. They define the goals that have to be met, and as it turns out, defining those goals is a very important part of these challenges because you have to tell the engineers what it is they need to be able to do before they

can build the stuff that does it. If if you have a very vague description of what needs to be done, the possibilities of achieving it are so varied that it can often paralyze a project nothing gets done. But if you make it very specific, then it narrows down the options and it gives more focus to the project. So the DARPA Robotics project itself was inspired by a real

world disaster, specifically the Fukushima disaster in Japan. Now, this was the nuclear facility, the nuclear reactor that was flooded and then suffered some massive problems because of damage to the infrastructure. And we're talking about a nuclear facility, Radiation is a factor, so it's a very dangerous environment to

send responders to. Dark As approach was to create a scenario in which a robot would need to go through an environment similar to that that was at Fukushima and be able to perform the tasks necessary to help avert true devastation and catastrophe without putting people at risk. So the idea is that you can use a robot which is not going to respond to radiation the way a living organism would and be able to actually carry out the stuff you need to prevent a really huge problem

from getting worse. So uh, some people have referred to the Darker Robotics Challenge as the Robolympics because it was a series of tasks that a robot has to complete and it had to all be the same. Robot teams could not build individual robots designed to complete specific tasks. If they could, it would be way easier of a challenge because most of the robots that are in use

today have very narrow parameters. There's just maybe a single task, or perhaps just a couple of small, you know, a small range of tasks that the robot has to do. It doesn't have to do anything outside of that. So when you're designing the robot, you just say, all right, what design elements will allow the robot to do what it has to do. Anything that's outside of that is unnecessary, we will not do it. Take the roomba as an example. It's a great example. It's a robot the vacuums that

cleans floors. Anything that's not necessary for the roombot to do that is put aside. You'd want to make sure that all the design elements in your robot complement its purpose. But if you have to complete a lot of different tasks, and those tasks are not always similar to one another, you complicate the whole design process by incredible factors. It's hard to even express how much more difficult this is. Now, if you've watched any of the videos that came out

of the Darker Robotics Challenge. You probably saw at least one or two that were montages of robots falling over and it's a little funny to watch. You see these big robots just topple over sometimes apparently for no reason. They just they just they're standing and then they collapse.

And there's something comic about that, you know that you can't deny that it's funny, but it also demonstrates that these robots are trying to do something that while we as human beings might find easy, it's a real challenge from a robotics standpoint. So this Darker Robotics Challenge had three phases. The first one was a virtual challenge, which in which teams were to design software that would allow a virtual robot to complete um certain tasks within a

virtual environment. So there were no real robotics here, but it was a test to see if if the teams could actually build the software necessary to make the robot do the tasks that needed to be done. The next had a trial where it was a physical trial where teams actually had to uh place a real robot through to you know, go to the next level. And then you had the finals, which were the ones that have

happened most recently. Uh, and I'll go through each of the tasks that they had to do in just a second. If we look back at the Virtual challenge, there were twenty six teams that were part of it to build that software. The robot they were controlling was a virtual Atlas robot, which is a humanoid robot and was a popular choice for a lot of the teams. When they moved on to the more physical trials, they still would

use the Atlas robots. The the challenge, by the way, did not require the teams to build their own robots from scratch. They could use pre existing robots, but they had to program them so that they could complete the necessary tasks. So not every robot out there would have been a good fit for this. Let's go back to the Roomba. The roombo would have been a terrible choice to enter into the Darbo Robotics Challenge. We'll be back with more of this classic episode of tech Stuff after

this quick break. So, uh, here are some of the virtual tasks that the virtual robots had to complete using this software. They had to walk through an area of uneven ground that had debris included there, so simulated debris had to have a software that would allow the robot to maintain its balance even while tracking a path to a specific destination. Through this this uneven terrain, they had

to attach a hose to a spigot and turn a valve. Uh. They also were given some artificial limits because and an emergency situation, you cannot always count on your communication lines being perfectly sound, such as that the Fukushima disaster. You need to be sure that your robot can contend with the fact that there might be lag between when you can issue a command and when the robot is ready for the next command. UH. That meant that some of the teams started to really look at ways to create

a semi autonomous robot. So some of the teams built robots that were capable of taking on some tasks autonomously. UH. Some teams avoided that entirely. They focused on using the robots as a direct extension of the control roles that the humans were behind. So, in other words, you might think of those robots as enormous technological puppets. The puppets would respond to direct human commands and wouldn't be able to do anything on their own. They didn't have any autonomy.

There are other ones that you know, you could have semi autonomy in the show it a set of stairs and you send it the command to climb those stairs, and it could do the rest. It could calculate how high it needed to put its feet and how to shift its weight, that sort of stuff. UH, And there were a lot of different strategies employed and to varying degrees of success. It wasn't like the autonomous robots automatically were better than the human controlled or vice versa. There

was actually it all depended on the implementation. So that virtual trial ended up having one team winning pretty pretty decisively. UM. It was the Institute for Human and Machine Cognition, which is UH, an organization in Pensacola, Florida, and eight other teams qualified for the trials that we're going to be held in December. Now, by the time those trials happened, some of the teams had merged UH and some other

teams were coming into it. Six of the teams would go on to compete at the trials that were held at the Homestead Miami Speedway, and they had eight tasks they had to complete that were similar to the ones in the Darker robotics final, and those tasks included manning a vehicle, um walking through uneven terrain, or otherwise moving through uneven terrain. Robots did not have to be bipedal. They could move around on however many limbs they had. They just had to be also able to do these

other tasks. Uh. They had to be able to climb a ladder. They had to be able to remove debris from a door and then opened the door. They had to be able to break through a wall. Uh, they had to be able to handle valves and to attach a hose to a spigott. The winning team of that set of trials was a group called Shaft spelled s C H A f T. This was a group out of Japan. Now, when this group was competing, they were

this little independent group. But then they got acquired by a little company that we've talked about several times on this podcast. You know him. You may or may not love them Google. So just before the trials, Google had acquired uh, this this this group, this organization. Now Shaft did really well, uh, which you know makes sense. I mean, you've you've heard the theme song. But out of thirty

two points that it could possibly score. It earned twenty seven, so pretty good score for for a robot and also was seven points more than the next closest competitor, which was from I H M C, the same group that won the virtual trial earlier. Third place was Tartan Rescue from Carnegie Mellon And that will be important in a second to when we get into the finals. Now, Shaft did not go on to compete in the DARPA Robotics Finals because Google announced that they were refocusing the team,

dedicating it to actual commercial Google projects. So uh, they were no longer being dedicated to this challenge. They were being dedicated to a real world product of some sort. Google's also had other involvement with some DARPA stuff, at least after the fact. Google does not tend to get involved in these challenges directly. Um, probably because of the military association of DARPA and Google is very careful to

avoid those kind of associations. Now, the finals took place over June five and June six and Pomona, California, and uh, it was at the Fair Plex and there was quite a crowd watching these events, and the reports I read were really interesting, and also the videos I watched were interesting because the crowd was extremely enthusiastic cheering on the robots, groaning every time a robot failed a task or fell over, gasping when that sort of stuff happened, because everyone wanted

to see these robots succeed. No one wanted any group to fail. And uh. Also, the other interesting thing was that it was really hard to tell if you were in the audience when a robot was operating autonomously as opposed to being controlled by humans. Um. That says two things, One that the line between these two is getting further and further blurred. And to that sometimes the robot would behave in a weird way, and you couldn't be sure if it was because the autonomous programming was lacking or

because the control mechanisms were limited. So, in other words, it's not that necessarily that the art has gotten so advanced that we can't tell the difference. It may be that the art has not advanced enough that we can't spot what is the cause of incompetence. And and I use the word incompetence mainly as a joke. It's just a joke because truthfully, it displays how difficult robotics as a discipline actually is how how challenging it is to design a robot that is capable of doing the same

sort of things that humans can do. It shows that the things that we take for granted as being pretty simple are fiendishly difficult when you get to a design and programming level when you're actually building a robot that's going to be capable of doing the same thing. So here are the eight tasks that the robots had to complete, and one of these tasks was called Surprise. I'll get

to it. But the reason it's called surprises that DARPA told all the teams, your robot will encounter a task similar to these other ones that you already know about, but we're not going to tell you what it is. And it meant that the teams knew that there was going to be something on that list of tasks that was not not defined, and that the robot would have to be able to contend with it in order to get a point for that particular task. And so the tasks were um there were eight, Like I said, eight

of them. You were awarded a point if your robot could successfully complete that task. So the score the final scores once everything was done were determined by how many tasks were completed successfully and what was the time of the robots performance. Now, when they were first designing the robots, teams were told they would have half an hour per task. So you have eight tasks, half an hour per test, that's four hours total. But the actual competition they were

told they would have one hour to complete all eight tests. Um. And why you might think that, how is that fair? As a as a bait and switch, you also have to understand this was all about simulating an emergency response. So under those emergency conditions, you can't expect to ask for more time. That's not realistic. Uh. And it added an extra layer of pressure on the teams. So those eight tasks, the first one was to drive, so the robots had to drive a Polaris Ranger XP nine vehicle.

These if you haven't seen them, they look like sometimes you might call mcgator or you might think of it as a golf car on steroids. These are um vehicles that are similar to golf carts. They're they're largely open, They've got heavier wheels than golf carts, do more warm horsepower than a golf cart would, but it's definitely in that range between golf cart and real car. Teams had five minutes to alter the vehicle without using tools to

make sure that their robots could actually operate it. So what the robots had to do was drive from the starting point to a destination, and it was only considered a success if the vehicle had completely moved past a

finishing mark. The vehicle also had to go through essentially a driving course with obstacles and cones set up, and it was determined that if the robot were to collide with one of those obstacles or one of those cones and cause it to move as a result of that collision, the robot would not receive a point for that ask the robot. Also, the team could choose for the robot too, instead of driving, to walk to the destination, but in that case they would not be awarded a point for

that task. They would forfeit the point um. But ideally the robot would be able to operate the accelerator and the steering wheel and maybe even the break and the shift. The cars were already or the vehicles were already started, The engines were already running, and they were already in high gear. Because it was considered to be the smoothest way for the robots to to maneuver the vehicle. But if teams wanted to, they could design a robot that

would be capable of shifting gears. It was not a requirement, it was just something they could do if they wanted to. At any rate, the robot needed to be able to drive safely from the starting point to the destination and then halt the vehicle either by letting off of the accelerators so it coasted to a stop, are actually applying the brake. The next task. That was just task number one. The next task was called egress, which is really just

getting out of the car. It sounds incredibly simple, and for humans, for most of us, it really is pretty simple. You know. We we intuitively know how to maintain our balance and to shift our weight so that we go from a seated position to standing. But that's not the case with the robots. You have to design the robot to do that. You have to program the robot to do that. You know, the robot be able to anticipate what a shift in its weight will do, what how

it's momentum will carry it forward. As it turns out, This was one of the trickier tasks that teams had to complete to get out of a vehicle. It was not an easy thing to do um but they were told that they could get out of the vehicle in either direction. They didn't have to exit outll the left

side versus the right side. That the robot could get out of either side, and again the task was considered complete if the robot could get out of the vehicle and maintain its balance and move on to the next challenge, and if they could, it would receive a point. The next one was door. Now egress was tough. Door was surprisingly tough. You would think that opening a door would be a pretty simple task. This was a push door

operated by a lever style handle. You had to push up or pull you know, pull up or push down rather on the handle and then push the door to open it, then step through the door. The door did not have a threshold, so there was nothing that you

had to step over. However, the door was a standard thirty six inch wide doorway and with the door jam that's closer to thirty three and a half inches, and some of these robots were too wide to walk through the door, you know, heading with their torso facing forward. They actually had to turn sideways and then shuffle through the doorway. And there were a lot of robots that fell down at this stage of the the challenge. They would lose their balance either when pushing the door or

when trying to maneuver through the doorway. One robot fell down at this stage and was able to pick itself back up very very slowly and deliberately, which is remarkable because it's the only robot that managed to pick itself back up after falling over, and that was the Carnegie Melon robot, the same one we mentioned in the Trials part in just a few moments ago. But anyway, that was it. You just had to open the door and walk through the doorway, and that was the end of

that task, and yet it was deceptively difficult. The next one was Valve, which was not about the game company, but rather about turning a control as if you were turning a valve to operate a fire hose or to shut down water to a specific part of the system, which makes sense. In a nuclear facility, you might have

to shut off or open a valve. In that case, the valve had to be turned counterclockwise in three sixty degrees in order for the task to be complete, and the team was only told that the valve would have a size between four and sixteen inches in diameter, which meant that you had to create a robot that would be capable of gripping anything within that range and then turning it in that counterclockwise direction three sixty degrees. Again triggier than it sounds. We've got more to say in

this classic episode of tech Stuff. After these quick messages the next step. The next task was called wall, in which a robot had to pick up a cordless drill. There were two of them available. They were not automatically on. The robot had to operate the cordless drill and and it on by squeezing the trigger, and use the cordless drill to cut through some drywall in a shape that

was drawn on the drywall. The drywall was a half inch thick, and the robot had to hold the drill, operate the drill and move in this shape and then remove the rubble. The idea being that sometimes the robot might have to cut through a surface in order to get access to certain controls uh that might otherwise be blocked by collapse of a room or something along those lines. Also pretty tricky, you know, using tools that were designed

to be held in human hands. You know, humans, we we have the ability to detect how much pressure we're using and to um to change that based upon what's happening. Robots a little more tricky. I mean, you can have sensors that alert the operator what's happening to what's happening, but it's not intuitive. Again, you have to program it. After wall was the surprise. Now, the surprise was something

that DARPA could change out from day to day. There were two days of this series of challenges, and I know that on one day, I'm not sure if this was the same for the other day, but on one day it involved picking up a plug and plugging it into a socket. Uh. I think they probably changed it for the second day, But I never could find out

what the surprise was on that case. But at any rate, it also ended up being pretty tricky that I saw one robot that attempted several times to plug the plug into a socket and it was sad and funny at the same time. But yeah, again, it was one of those things where it was similar to tasks the robots had had to do previously, but wasn't something they were expressly told they would have to do during the actual challenge,

so that's what made it really hard. The next task was rubble, in which the robot had to walk through a debris field or on top of the debris, either through it or on top of it. Again, very challenging for robots to maintain their balance. There were lots of you know, Boston Dynamics robots that were pretty good at doing this, but still pretty tricky. You saw a lot

of robots fall over at this point too. And then there were stairs that was the final task was to climb us up stairs that had a rail on the left side but no rail on the right side, And once the robot was completely on the top of the stairs, it was considered to have completed the task and the course and it's time would be logged. So again, those those tasks, for the most part, pretty simple for your

average human to do. But imagine that it's your job to build a robot that could do all of those things. It has to be able to have some form of perception, has to be able to see either for the human operators to pick up various two tools, or navigate through certain areas, or for it to to see and operate autonomously and has to be able to perceive its environment

understand depth. In fact, a lot of teams had trouble with the the wall task because it was very hard to perceive how far away the drill was from a hand to pick it up and then use it again. Things that are pretty easy for most of us very hard from a robotic standpoint. Um. And like I said, the door was one of the hardest ones. Uh. A lot of robots had a lot of issues just walking through a doorway, which is both funny and and really does bring to light that we've got a long way

to go with robotics. Um. Well, if if you, as someone in the crowd had said, if the robotic uprising happens, just close the door and you'll be fine. Uh, Because it's tricky stuff. It really shows the automation is. We take a lot of it for granted because the examples we see tend to be pretty elegant because they're designed

for a specific purpose. But as we start looking at a more general purpose robot, we begin to understand exactly how complicated we human beings are, so to design a machine that can operate within our human world seamlessly is an incredible challenge. So who won, Well, the first place went to a South Korean team k A I ST. It had its robot d d r C Hubo, which completed all eight tasks in forty four minutes and twenty

eight seconds. Those eight tests forty four minutes and twenty eight seconds, remember that's driving from one point to another a very short distance, actually getting out of that vehicle, walking through a doorway, cutting a hole through a wall, turning a valve, plugging in a plug, um, walking over some debris, and climbing some stairs. Something that would take you know, a healthy person. If it took the more than ten minutes, you would wonder what was going on.

So another example that the winning team did it in

forty four minutes and twenty eight seconds. There's also a great preparation video for the d r C Hubo that was put out by the South Korean team that I loved because it was like a training montage from Rocky and and they It was clear the team had a sense of humor about this because it was showing the robot not just complete tasks that would be similar to the ones that were in the actual challenge, including things like climbing a ladder or um or lifting a tool,

but also doing your typical, you know, training montage stuff like doing push ups or taking a fighting stance like you're a kung fu master, And that really tickled me. I thought it was pretty entertaining but also very impressive about what they were capable of doing with that robot. The second place team was I H M C. That Pencacle of Florida group we talked about a minute ago. They were using the Boston Dynamics running Man Atlas butt Um. It took it took about fifty minutes in twenty six

seconds to complete all eight tasks. After he did so, after it had climbed the stairs, it lifted its arms in victory and then fell over, So I guess that's Hubris. The third place team was Carnegie Melon. Their team Tartan Rescue, the one that had come in third place in the trials as well. Their robot was called Chimp, which had

very long arms. Is a red robot with arms that looked kind of freakishly long if you compare it to a human, and it also had treads like a like a treadmill type thing on both of its legs and on both of its arms. And that was the robot that fell over in the door challenge and was able to right itself by itself, the only one that could get up by itself and didn't need a team of humans to actually lift it up. Now, those were the only three teams that actually completed all of the eight tasks,

and they won the prizes. First place was two million dollars, second place was one million, and third place was five thousand. The other teams broke down like this. Four of them were able to complete seven of the tasks, only one was capable of completing just six tasks. You had two teams that completed five, two teams that completed four, four teams that completed three, two that completed to one that completed one, and four teams weren't able to complete any

of the tasks successfully. Um. Now, the whole point of this was to really push that state of the art forward, to have engineers think, what would you need to do to build a robot capable of actually responding to real world situations? Who are the challenges that are in the way of how can we advance the technology both in the hardware and in the software to overcome these challenges. And the goal was not to build a super robot that's going to save the world. This is not you know,

the Avengers age of Ultron. That's not what's going on here. It's all about incremental improvements in the art and discipline of robotics so that the next generation of robots can benefit from the research and development done in this first generation and a good way of seeing how this plays out in the real world, because you know, you might think, well, that was a clever demonstration, but what what can we

expect here on our day to day lives. Take a look at how autonomous cars are coming along, because of course, DARPA held the Grand Challenge back in the mid to late two thousand's, and that was the challenge in which different groups tried to build self driving cars that could complete a core whether it was on the desert or

simulated urban environment. And we are now seeing people who worked on the various teams that that competed in that DARPA Grand Challenge building what will be the first generation of driver lest cars that will eventually make it to the consumer market. Google has people on its team that competed in those Grand challenges, So we're seeing that expertise that was developed as as engineers were given a problem

and told find a way to solve this. We're seeing that expertise they developed come into play in the real world now. And it's sure it's gonna be a few more years before we get autonomous cars as a reality that an actual human being can go out and purchase, as opposed to a representative of a company or you know, just uh, you know, someone who is there to demonstrate the viability of the technology. It's going to be a few more years before this becomes something that you or

I could go to the dealership and actually purchase. But the reality is there on the horizon. Same here with the Darba Robotics challenge. It will likely take a decade or more to develop a humanoid robot or a robot capable of operating within a world designed by humanoids for humanoids and do so seamlessly. It will take a lot of time and a lot more development, but the foundation has been laid, so I'm particularly excited to see what

the future holds for us based upon these results. And while those montages of robots falling over are really funny. I am actually optimistic about what robotics will be able to accomplish in the future. I'm just also realistic that it's going to take time. It's not We're not on

the verge of a terminator like future. The state of the art and artificial intelligence has gotten gone very far in certain realms of computer science, but when it comes to robotics, there's all there are a lot of problems that are still very difficult. You know that vision is a big one. H and again, just being able to maintain balance again one of those things that we take for granted, really a challenging problem. I hope you enjoyed

that classic episode about the DARPA Robotics Challenge review. If you have suggestions for topics I should cover in future episodes of tech Stuff, please reach out and let me know what you're thinking, because otherwise I'll never know. The best way to reach me is to use Twitter. The show's handle is text Stuff HSW and I'll talk to you again really soon. Text Stuff is an I Heart Radio production. For more podcasts from my Heart Radio, visit the i Heart Radio app Apple Podcasts or wherever you

listen to your favorite shows. H