Hello, everyone. Welcome to the next episode of Hewlett Packard Lab podcast from research to reality. This time, we are doing the spotlight on animal tracking. I have amazing pleasure and honor to host two gentlemen here in the room and one gentleman out of Africa. let's start with a remote. attendee. Alex, would you mind introducing yourself? Yes. Good evening to you. Thank you for having me. It's it's a great pleasure.
I, I met, HPE, about two years ago with the idea of creating this app that would automatically be able to identify animal tracks and, and prints and, I'm happy to say we are a long way down the track and the signs are fantastic. And it looks like we are getting this right. Thanks, Alex. I'm sure you'll be able to tell us much more about it. Let's introduce, first. Shivang. Hi. Shivang. Hi Dejan. Nice to meet you again. yeah. Hi, everyone.
I'm Shivang and I. I'm a researcher at here in the Networking and Distributed Systems Lab working with, Puneet Sharma, our director. I've been here for about a year and a half now. before that, I was finishing up my PhD at Northeastern University, where I did my PhD in wireless networking. Thanks, Shivang! Hi Lianjie! Welcome back on the podcast. It's a pleasure to be back, Dejan. my name is Lianjie Cao. I'm a senior researcher at Hewlett Packard Labs.
My journey with HP slash HPE started with an internship back in 2013, and I joined Labs as a full time researcher in 2018 after I got my PhD degree from Purdue. Okay, excellent. So I heard about this project of animal tracking. I always wanted to learn more. I never had a chance. So this is, you know, like, a double opportunity both to talk to you guys and to learn a little bit more. So who wants to, take the first stab at it? Yeah, I can start.
so, yeah, this project is basically, quite an interesting project, as you mentioned yourself. so at a very high level, it's basically about trying to digitize and save this ancient art that is called animal tracking. And and what is animal tracking? Basically, it's this ancient scale that was evolved in Africa thousands of years ago.
And basically what it involves is these human trackers, they go out into these national parks, forest wildlife reserves, they must form a mental image of what the animals are doing, where they're going, and that they don't do that by befalling the animals. They do that by looking at the signs that the animals have left behind. from that could be their pawprints. That could be droppings or other scratches that they leave behind. and so that's what, animal tracking is.
And unfortunately, due to, urbanization that's happening globally. Right. this ancient art is sort of withering away. And our goal here with Alex is to try to save that ancient art. Right. And we are building an end to end system for that. I'm sure we'll go into details, later, but just briefly. We have a front end. We have a front end piece, which is the mobile apps and the web apps and then which help us gather data and get images from the field.
And then we have our back end, which has host and the intelligence. Right. The machine learning algorithms and whatnot, to then take in as input those images and then output which species they belong to and so on. So what was your first thought when you heard about it? Lianjie, about Alex and his project, which really is amazing. Shivang can start first because I joined the project a little bit late. I can add to that. So yeah.
So, yeah, for me, I remember Puneet one day just came to me, and talked to me about this project, and at first I was a little confused, and, like, I didn't know what to expect from this project. but then, thinking in the whole sort of, scope of the project, I was very excited, basically, and trying to see what all it entails. And it's, you know, very different from typical research projects that at least I've taken or over in my research career.
And, and also when we we started speaking to Alex for the first time, it really clearly came across how passionate he is about the project. Right. And that itself is very motivating. Right? how passionate he is, how inspiring he is. And that helps us push ourselves further too. I mean, to me, it's indeed a very different project, a different project, when compared to the self entertaining kind of research project that I do in labs.
and I joined the project a little bit late, mainly working on a machine learning part of it. and, but the first time I heard about the project, like, oh, man, finally we're shifting our research directions a little bit and starting to work on some, something fun. And I'm ready for an adventure in the jungle. Right. So. But the thing is, like, of course, we never do that.
And, it's really a great and exciting, project, like Shivang said, and a terrific experience to work with experts like, like, Alex from a completely different field. Okay, so we've been talking about Alex this. Alex that. Alex. How did you approach HPE? Okay, so I run an NGO, a not for profit company in South Africa called the Tracker Academy, and we train young rural people from big wildlife areas in traditional skills of animal tracking.
A one-year formally accredited program, we are a school, and then we deploy our graduates into conservation jobs. And it's just become evident over the last few years how the ancient traditional skills of tracking in Africa have diminished. We are losing these skills and waste. And at about the same time I met with, Dobias van Ingen from HPE, he, from Aruba, and he was in South Africa on a conference and we happened to meet just coincidentally.
And I told him about the idea that we needed to digitize this ancient data before we lose it completely. And and he took the idea back to HPE. And that was two years ago. And here we sit now with, I think it's version three of the app. and we are a long way down the road. And I must say, there's been great success and it's been such a pleasure to work with such an incredibly talented HPE team and the HPE labs. What these people have been able to achieve is just astonishing.
So I can thank you Alex I can hear all this excitement from your side, but I'm curious from far away Africa, how did it excite you here in Milpitas? Lianjie and Shivang. Sure. so I think to me it's the impact. So we have been working on an innovative idea. like challenging problems in the labs, but those ideas and those, challenging problems usually take a very long time to see the impact, like, for years.
But this project is very different in the sense that we can directly, almost directly see the impact immediately. And all the improvements and all the enhancement we made in the system are going to be used by Alex and his team on the field, like right away. So there's no way do you see the impact? And I think also the impact of sort of like a beyond labs, even beyond HPE. Like we have been always saying accelerating the impact, being a force for good.
Yeah. And I think, to add to what Lianjie said, he mentioned a bit about the social impact. But, I mean, even from the technical side, as researchers, right, we get excited about tackling challenging problems. And this, again, I did not you know, we have we should expect challenging these sort of challenging problems at the beginning. But as we went through the project, we encountered several challenges and technical challenges that we had to overcome.
And as researchers, nothing more exciting than that, right? Yeah. So I can see why historically this skill is very important. But how is it still important today? Alex. Yes, that's a good question.
Today, in modern day conservation efforts, we need trackers to track to collect data on behavior of animals and the trackers, but possess the knowledge able to to accurately identify and interpret the signs of the animals left by animals and thereby able to collect important data that otherwise would not have been collected. that trackers are able to know what animals are doing even when they're not there, by by following and interpreting their signs. We also need trackers in anti-poaching.
I'm not sure if you know, but there's been a crisis in southern Africa of rhino poaching. We've lost something like 70% of our rhino population in southern Africa over the last decade and a half, and we need talented trackers who can pursue the poachers on foot and track and find them.
So it's, the trackers are the unsung heroes, and their skills are becoming increasingly important as as we go through time and as conservation management start to realize the value, the tangible value that this ancient skill brings, the modern day conservation efforts. So I can understand people in the field, and all the challenges they have. I can imagine them. But what are the challenges for you here in Milpitas? Shivang and Lianjie, I can start, like.
So our goal, the HPE team is to build a digitized solution for Alex to solve all the challenges. he mentioned. And the the challenges for building such systems is, like, every step of the way. So if you think of this, like starting with, we need to figure out the design of the architecture for this end to end system, from capturing the footprint in the jungle to infer the images of the species, in the back end. Right.
So, for instance, we need a front end to capture the footprints and annotate the footprint with the information related information before they get lost. And we also need to upload them reliably, to the back end. So all those kind of things we need to take care of even in the back end, we also need to think about how should we clean the image, how should we store them, label them and then infer the the species of the footprint, all those kind of stuff?
Yeah. And to add to that, I mean, we have way more challenges because of the fact that the geography of the, the problem itself. Right. A lot of these trackers are working in remote areas where there is like little to no connectivity at all. Right. And we are asking them to upload these images from those areas. So how do they do that? There are all sorts of technical challenges that arise because of that lack of connectivity. Are there any non-technical challenges?
Alex. Because here we only jump on technology. Technology. I'm sure there are other kinds of problems that you are, exposed to on a daily basis. Well, yes. In in training the machine, the algorithm, we need to collect many, many examples. Let's take for example, we want to train the machine to identify a lions track. We need to to, to up to identify, photograph and upload at least a thousand different examples of lion tracks. And so that takes time.
And you so and it takes people that who know how to identify a lion track easily. If we have less than competent trackers training the machine, it's, it's going to have a great impact on the on the outcome of the, the the performance of of the app. So we've got to make sure we've got expert trackers out in the field collecting literally thousands and thousands of images of 120 different species. So that's an incredible feat. And it's it takes time and weather often is a problem.
so yes, that's that would be the most, apparent challenge we are dealing with, but we're getting through it slowly. I think we've, collected about 30,000 images so far. So you gave me some hint that you are using an app to solve the problems. Can you tell me how you are solving the problems using this app? sure. So, for the app side, we basically have like two different types of apps. We have the mobile app and installed on the smartphone of the trackers. And we also have the web app.
So for the mobile app, the users can create different type of account like public account and professional account and things like that. And then in their account they can browse first all the images you have taken before with the different types of information embedded. And they can also, of course take new photos of the footprints and then at the same time associate the metadata information of the of the species and things like that.
And then and finally they can upload your images to the back end. And in the web app, the app, the web app is totally different. It's more for the purpose of analysis and management. So in the web apps, the administrator can go ahead and browse all the images uploaded by different users. Professional, public and all those kind of stuff. And they can check into the detailed information of them and try to, modify the information of some of them are wrong.
And what's more interesting is they can also browse the images, filter them and show them or, you know, the active map to see the geographical, you know, information on the map so they can get a better visual understanding of where the animals are spotted and all this kind of stuff. And in the future, we are also thinking of doing more deep, deeper kind of like analysis and understanding based on this type of geographical information.
For instance, we can try to learn how global warming impacts the animal migration at a very high level and things like that, but also like to highlight the fact that, I mean, the challenges that we describe this, such a diverse set of challenges. And so we needed a diverse set of people with diverse skill sets to tackle those problems. Right. So this effort is sort of a pan-HPE effort. Over the time we've gotten more and more collaborators from outside labs.
So for example, we have Ezmeral BU who are providing us with the back end to help us run the modeling and the inference on, then we have, the Asia Pacific Innovation Center team. Who is Who's helping us build the apps themselves. Right. Because, I mean, these apps need to be very professional. Grade. We can build some apps, but I'm sure they won't be as good as what these guys put out. I mean, these guys are really talented at what they do.
so yeah, that's just something I want to highlight I'm glad you started to talk about business. you have so far received help from various businesses like Ezmeral or Asia-Pacific teams. But in the big picture, how do you see this becoming sustainable? Not necessarily making money out of it, but someone needs, on a daily basis, annual basis to support this. So it should eventually go into some business, into some production. How do you gentlemen see this?
so I think, the idea who we are at the moment, currently we are relying more on the goodwill of, let's say, HPE and Alex that, we can sustain this project, in the short term. But I think in the long term, it can also be about once we if you want to monetarily sustain it can also be about how we gamify the app a little bit, maybe for like folks like you and me. Right. Like we can go out in the field, snap, you have pictures of footprints and send it up and get a result.
Right? Okay. This is a lion’s footprint. This is a wolf’s footprint. And then maybe that builds this sort of ecosystem of users that, the an end goal could be. And then that could ultimately help us sustain, what we're doing. And hopefully, I mean, we always will rely as, again, HPE's nature of being a force for good, right? I mean, that's always a thing that will hopefully help us sustain this in the long term. Yeah. And to add a little bit to that.
So this is becoming something that the the impact of this project is going beyond HPE because we have been talking to and Alex as well, talking to some other service providers, they are willing to provide like hosting services to the project because you want to advertise, you know, of course, they're on those and things like that. So those can become a more, you know, general coverage team across companies and things like that and be a very good showcase of HPE's technology.
It can be, can be applied to solve such problems. So we can get some, you know, some similar attentions and projects from potential customers. Any opportunity for our Aruba because they are our edge side. Yeah. So yeah, as I mentioned briefly before. Right. the work that a lot of these trackers are doing are in remote areas where there is basically no connectivity at all. Right.
so Aruba, basically us working together with Aruba can help us tackle that specific part of this problem where, maybe using technologies like satellite or private 5G, we can have some deployments in these remote areas. that can help get connectivity to those folks that don't have it. Right. And that will ultimately help us reduce the latency that these guys face in terms of uploading the images, getting results back from the servers and so on.
And if we can do that, obviously that will ultimately help the quality of experience that these guys, face in these remote areas. So accomplishing what you have, which is really outstanding, what was the most challenging about it, and was there anything unusual in terms of your solutions or surprising things?
I think to me the most challenging part is actually labeling the images, which is kind of like strange because Alex came to us for help, and it turns out that we need his help to label the images first before we can apply technologies like cloud computing and machine learning to help him. So it's it becomes some sort of like a chicken egg problem. So. Well, you want me to help you, then you need to help me first. That kind of stuff. Which is kind of surprising to me.
and if you think about this, like, what's worst in this case, in this problem, is we there is nobody else except Alex and his team can help us to solve this problem. Because these guys are professional for me. Like, I can tell the difference between cats and dogs, but I cannot tell the difference from the footprint. Right.
So and this is also an in from the other, perspective, it's like, there are always something we take for granted in our research project, but in reality, those kind of stuff like a label that you measure and things like that are probably not quite there And, Alex. Have you considered any standardization? I mean, there are different classes of animal tracks and all of that. Yes. Well, there's standardization in terms of trackers skill set.
we are able to objectively evaluate a tracker’s skill set in the various components, practical components of tracking. And that is what is established. And, the South African Department of Education recognizes, our standards in that regard. And there are standards in the US as well, run by other companies that that promote and do tracking assessments. But in terms of, standardizing the data, that's very hard.
And that's I just wanted to add the most complex part of this program, this whole project is that a lion footprint. No two lions footprints are the same. And it depends on the type of soil that it stands in or how it's moving, if it's turning, if it's running, if it's old or young, it's constantly throwing up different types of, of of forms of, of the same species track. And that's why we have to, get, get so many, capture so much data to be able to train, train the machine.
So that is, that is, that is the most difficult part of it. And you know, we there there are similar apps out there that identify, identify plants and trees by taking a photograph of the leaf or the flower. That's a that's a far simpler, procedure and project because there's, there's much less very variation between an apples leaf that grows in the north of the US and then in Europe, whereas in, in, within tracks, as I said, there's just a great variability.
And it seems intuitive to me that this is very ethical, what you are doing. But have you ever considered that angle of ethics, of animal tracking? Yeah. I mean, at the Tracker Academy we have a whole module on ethics because many of our graduates go into protected areas, government protected areas, and have to protect endangered species. Excuse me. And so they have to they have to undergo polygraph testing every, every six months. and their livelihood depends on being ethical.
and also from from another completely different standpoint, the, the, the observation of an animal's track or sign and thereby being able to interpret what the animal is doing is a completely noninvasive, approach. You don't have to go off to the animal and dot it and put a collar around it and caught it and give it drugs. Trackers are able to tell where animals are moving and what they're doing, as I said earlier, without even having to see them.
And and that's very much goes to this idea of sustainability, of low impact on the animals, not disturbing them. And I think that that certainly falls under the, under the umbrella of, of ethical conduct. Thank you. Alex. Coming back to Milpitas from Africa, what did you two gentlemen find most rewarding in pursuing this project, other than being extremely exciting? Yeah, I can start. yep.
So for me, I think, as you mentioned a few times already, it's just the direct impact that is there of the project. I mean, all the projects we do, I mean, we would like to think that they ultimately help society in some way or the other down the line. Right. But with this project, you can really see the directness of the impact. Like whatever we do, it goes into the hands of these trackers that are going out into the wild.
and ultimately been the goal, obviously, is to contribute to the well-being, of our planet. Right. Finally. That's what, we care about. So that's really the most rewarding part for me. Yeah, to me, like, I learned how to track animals, of course. And the other thing I also realized through the project, it's basically the gap between research project and real world problems.
For instance, the problems that Shivang said, some of them are probably not technically difficult to solve, but we do need to, to think from user's perspective wearing their shoes to identify and to realize the problem before we can do something about it, though. That's the thing I learned. And, Alex. And, Alex, why, does the audience why does the public care about this? Now? That's a good question. The public should care about it for a few reasons.
One, it is part of ancient African cultural heritage. and it has it certainly in South Africa, due to the effects of the apartheid regime and, and forcing people off their land, as well as the the rapid worldwide trend of urbanization, people have forgotten how to and lost the skills of tracking.
And what I have seen is that when we engage rural people, many of whom are unemployed, living on the outskirts of these big wildlife areas, some of the last viable wildlife areas left on the planet, and they stare jealously through the high wire fences. One way to capture the hearts and minds of these people is through their own traditions, that being the traditional skills of tracking. And so it has a twofold function for me.
it's it's a skill that can be used in a, in a technical manner to better protect, endangered species. But it's also a skill that goes directly to the hearts and minds of the people who are the custodians of these wild areas and these, and these endangered species. So that's why I think the public should be caring about this project. Great explanation. So we came close to the end, of this discussion.
Can we just close by, each one of you very briefly stating, what do you do when you don't track animals? Yeah, I can start like, I basically do hikes, like a long hikes like five miles. Ten miles. I play badminton from time to time. Used to play more often, but much less. Now I play soccer sometimes I also do a little bit woodworking at home. I have a small garage shop kind of stuff. Nice. Yeah. Thank you. Yeah. For me, I'm.
I'm getting a big sports guy, so I like to play a lot of sports, like, tennis, badminton, cricket. I don't know if people know about, but it's a great sport. it, and, I seen with, watching a lot of these sports as well. Then I like to attend, concerts of artists, music artists that I like, like watching stand up comedy. That's authentic. Thanks. What about you, Alex? I love I love Shivang's answer. I have two daughters they are six years old and eight years old.
They were both born at Londolozi Game Reserve, where I worked for 23 years. we've now moved to the town because of, their schooling. We needed a better school for them. But what I do do a lot of is take them into wild areas, and I'm trying to introduce them. Well, I'm not trying, but I'm introducing them to to tracking and tracking. It's just such a wonderful way to get people to immerse themselves in wildlife, to become connected with nature.
And I'm seeing the vehicle of tracking, doing just that. And I love spending time in nature with my two little daughters. Thank you very much, Alex. Virtual handshake to you and handshake here to Shivang and Lianjie. I really enjoyed this. I'm sure our audience will too.