Big Applications for Big Data

talk a little bit more about big data. You know, we talked about big data. It's okay, I'm going to use both pronunciations. Okay, fair enough, so big info, uh no, big data because we're all you are talking about computer information here and not the android from Star not the android from Star Trek. Yeah, just get that all the way.

So we're we wanted to talk more about applications of what what different organizations, companies, governments are using big data for what they're mining out of this huge amount of

information that we are generating every day. Now. You may remember in our last podcast we said that we're generating about two point five quintillion bytes of information per day and not just humans, but you know, sensors, things that are uh indirectly yeah, internet things, stuff that's connected to the Internet that we're not directly in putting data into. And for those of us who missed the last podcast, what's the difference between this paradigm of big data and

just say a lot of data. Big data? We're talking huge, huge, enormous amounts of information. When we're talking two point five quintillion bytes, that's half of all the spoken words that humans have uttered since the dawn of language. So we're talking about in two days, you generate as much information as all the words we've ever spoken ever of all the people. So there's this issue of volume, but also

these characteristics we talked about, like velocity and variety. So it's not just the amount of data, but it's interacting exactly extremely fast. We're gathering it at an incredible pace. I mean, you're you're gathering data at an unprecedented pace. You are. It's rich and intense and everywhere. Yeah, and it's all different types of information. And also there's a fourth v that we can mention, which is voracity, which is the quality of the information truthiness. Yeah, yeah, there

you go. Uh we're gonna tell you some strategicies that people use with big data. Uh it does mean truthiness. I mean it means that the data is essentially high quality. Right. Yeah. Voracity is really kind of their way of saying, how good is your information? And sometimes you don't know how good your information is, because again, you've got a lot of it. Until you analyze it, you don't really know

if there's anything useful there. Uh. IBM is in the business of of leveraging big data and helping other companies leverage it. And uh and and they say very clearly on their website and in their white papers that the important part here is that you have to figure out what your goal is before you start just looking at big data and saying we need to be part of this. Because whatever your goal is, that's going to end up informing your approach to using that information in a way

that makes sense. Otherwise, you're just talking about an enormous resource that may not be directly useful to you. You're just kind of looking at it and thinking, I want to make use of that information. I just it's too big a problem for me to even get a grasp on how I want to use it. So you can't just run at it and say, give me some ones and zeros, I'm gonna make magic happen. You have to have a plan in place first. But I've got a lot of really clever people have figured out really cool

stuff to do with this information. Sure. In the last podcast, we talked a little bit about traffic analysis, which was a very kind of you know, you know, it's an easy to understand application of big data, right, So let me give you an example using Google's approach. So the

way Google would generate traffic on Google Maps. If you were using Google Maps on a mobile device and you wanted to try and get from point A to point B, and you wanted traffic to be part of that that equation, that route, then what it would do is it would

start looking at information of other Google users. Uh. There are other systems that use this, the Dash system, which doesn't really exist anymore, but uh, it used a very similar approach where it would send anonymous data about vehicles that were moving through a particular region and the speed at which they were moving. It was just sampling the vehicle, yeah,

sampling the vehicle's location. It would get the GPS coordinates and it would just sample it over a certain amount of time and derive how fat that vehicle was moving down the street based upon the information, right saying, okay, well it was at this point at this time, and it was at this other point a little bit later,

and this other point a little bit later. Therefore, that means traffic is moving at this speed down this particular street, and then it's extrapolates that sends it out to everyone so that you know which routes have the heaviest traffic. Now the Yeah, it's it's a very simple approach to big data in the sense that it's just taking real time information, analyzing it, and sending the results back very quickly. It's not storing information, it's not trying to be transformative

with the information. It's just trying to make sense of all these different pieces of information that are coming in and then making it meaningful to the people who are using the service. So that's one example, but that's just one. You can actually see some pretty interesting patterns when you get huge amounts of information. You can see patterns where

you might have thought there was just chaos before. So you you can look at a system that you might say, well, from the outset, it just looks like stuff is happening. But now when I see all this information, it's broken down like this, I can actually see trends where before I just saw stuff. So education is a good example of this. So let's say you're a teacher and you're teaching a class, and you have your classes submitting schoolwork in a into a system that can then analyze the

school work. So you're grading the kids. You you might actually just be inserting the grades into the system. In fact, it may not have any connection with the kids directly at all. You might just be the teacher in the system. The system would be able to, if you're using a very sophisticated approach, be able to start detecting trends in each individual student's progress. So you might be able to say, oh, while while student A isn't failing, the trend indicates that

the student is beginning to struggle. So I need to adjust my way of reaching this student so that I am not leaving the student without any support. Oh, I wonder, so would that involve comparing uh little signals with millions of other students, Like if we we've seen that when these things start to happen, statistically, that means like we're

heading towards failure. It would mean that It would also mean that on a larger level, you might see that an entire classroom is having some issues, which would tell the teacher I need to change my approach. I I you know this, this concept that I've tried to teach. Obviously this has not worked out. So I need to find a new way of getting this across in a way that makes sense to my students, or perhaps help an entire school system figure out how to how to

grade and test better. Exactly. Yeah, you know how if you take a survey um and your survey has just a hundred people in it, well, it's probably not very representative of the entire population, but the big but you can play family feud, right if you have a thousand people, it's better. If you keep increasing your sample size, your statistics become better and better at representing stronger trends that you want to look for. And this is the same

thing you'd see. It's why big data is great. Right, you're increasing your sample size, right, Yes, As you increase that sample size, then you can actually start to recognize things that are truly trends and not just a one off. They're less likely to be anomalies. Yes, exactly, And this

has tracked a lot in especially in consumer segments. I mean, you know, like every time you buy something on Amazon, it it collects a little group of other things that people who have bought that thing have also bought in case you want to buy that thing, um and and can be very useful not just for not just for that basic you know, like we want to sell more stuff, but uh yeah, I mean with Amazon sometimes it's creepy

how how on spot it is? Or it's creepy how not on spot it is, and you wonder if they're something you don't know about yourself. My my favorite, my favorite is that if you start to find the products that are on Amazon that have the ridiculous reviews, like the ones that are just you know, it's the product itself is absurd for or the gallon of milk there they're there are examples of products that are on Amazon that people have written like novellas in review and the

novellas are hilarious. Like you get to a point where it's like it's like a soap opera that opens up and it is this whole thing, and then the last line will be some throwaway review of the product or whatever, and it's it's I mean, I love it for the absurdity. What's interesting is that the related items always tend to be the other ones that have similar ridiculous reviews, which means that even then always tend to include horse mask.

Horsehead mask. You know, yeah, people like you bought this thing, right, but at any rate, the I wonder if you click on that, if you also get like the Godfather as a suggestion, you've seen horsehead mask? Right, I've seen horsehead mask. Yes, I've used the internet, so yes, I have seen it.

But yeah, that is another example. Right. Amazon uses this in order to make more sales, because the thought is, if you are interested in this one particular kind of product, then you're probably interested in these other products, especially if there's a history of other people having bought those together, or at least you know, I bought them at some

other point in their and their history. And then you can you can start pouring that information not just into what are people going to buy next, but into why are people buying this thing, and start tracking things like flu outbreaks. Right, Okay, so this is this is This is one of those things that I thought was really, uh an interesting example of using big data in a

way that you wouldn't necessarily first think about. This was something that Google wrote up a white paper on there's actually a full paper about Google using information to detect influenza outbreaks. And the way that they did it was they essentially found search queries that people were putting in that indicated that someone was feeling sick, especially things like

various symptoms and stuff. And then by relating that information to specific regions in the world and seeing multiple people requesting this information from say a particular city, they could say, this looks like this is an outbreak of the flu. They said that there was a reporting lag of about one day, so a day after a certain you know, a large enough sample size of people are looking for this, Google could say there's a potential flu outbreak in this

very specific area of the world. Maybe we need to you know, by a learning something like the Center for Disease Control the CDC, They could say, we need to head this off before it becomes some sort of pandemic, right, which basically says to me that that that because of Google and big data we're going to be able to uh prevent the inevitable zombie. But exactly the scary thing is will probably never even find out about it, right because of stuff like this. They'll get to the government

before the public knows. No, there there has to be enough queries of dead uncle trying to eat my face for at least a few people to take notice. If you happen to be checking Google trends on dead uncle turned on Twitter long before. Yeah, you know, just the tumbler alone would be But this is funny. Actually, Google trends is a great, really simple, really straightforward example of how big data is really interesting, like looking at um the popularity of a search term over a period of time.

I mean, it's so cool. You can get lost in these bore to see is watching the spikes when you know different movies or books or cultural events happened. Yeah, when Kanye West does something embarrassing and immediately shoots to the top. One I love is when you can look at the historical ones because they have every book ever written, right, They've scanned that in and then they can you say, historical ones. I just said, they're thinking like bad play.

My American cousin just think like Google trends throughout the centurial. You can look at you can look at like back to eighteen hundred in the books they've scanned, right, um, and you can you can chart changes in spelling. Right.

You can compare spelling a word one way versus spelling it another way, and watch one go down with while the other goes up one one data artists that I'm going to talk about in a little bit created a graph of the use of hope versus despair and in recent years, just just watching the times when when despair overlapped hope. Interesting, beautiful. I can't wait to talk about that. Well,

and and you know, let's start with other applications. Well, I was going to mention that, you know, when you think about Google, Google's mission statement is all about big data, because they're about organizing the universe's information, which when you think about that, you know they want to index and organize all the information everywhere that we ever encounter. That

is big data. That's that's as clear as you can get. Yeah, And so the fact that they're able to demonstrate the usefulness of this proves the the the the utility of their company, right because otherwise if if all they did was index this and there was no uh, actually useful, Yeah, then you'd be like, well, this company is just not going to stick around. So besides education or predicting a flu outbreak, you can actually use it to monitor cybersecurity

and check a network's health. So if you see a spike in network activity, you can check it out and make sure that it's not a d d o S attack, a distributed denial of service attack, so that a hacker hasn't said, hey, this website has raised my ire, I shall direct my zombie computers to attack it. UM. You know, being able to see that kind of stuff and respond

to it in real time is really useful. And you obviously need to have a robust, robust system to deal with a lot of information because it may just be that it's a heavy amount of traffic for completely legitimate reasons. So that's another implementation of big data. UH. It's also part of what they're talking about when they talk about the smart grid UM for electrical electrical companies to be able to get energy to the right places at the

right times and prevent brownouts and blackouts. UM system overloads. People have talked about using UM tracking the number of ups packages sent to track how well the economy is doing. Interesting. Yeah,

I've definitely heard about the smart grid stuff. I mean, there are a lot of utility companies that are running at close to full capacity, and being able to to see where a demand is going to be at any given time means that you are reducing the demand on any individual power company because they can all work in concert together and that way you don't have these you reduce the possibility of a brown out or a blackout. Um. So, I mean that's clearly important, but that is a lot

of information. You're constantly getting feedback from all the different meters essentially smart meters, and even if you get down to it, you can have smart appliances that are very specifically giving both you and the network more information about power consumption. So that's all also important. Weather forecasting another

important part. Talk about gathering all the information from weather sensors around the world and looking at the information and detecting patterns because our forecasting abilities, don't know if you noticed, not so great. Sometimes it's hilarious because when you think about it, we have so much power and technology devoted to produce acting the weather and sometimes we're still so

it's such a complex night. The other night, the hourly weather on online was telling me zero percent chance of precipitation. We had a storm that was knocking limbs out of trees. It was like you couldn't see ten feet for the rain. Yeah. Yeah, And it's hilarious when you take a look and say, wait, they forecast ten days out, How how reliable is that

tenth day? There wrong about what's happening right now. Well, and beyond that you have things like uh fraud detection, and also governments can use big data for tax collection purposes, looking at trends in taxes and the way that people are paying taxes, and maybe comparing the way people are paying taxes versus what they supposedly, oh in Texas, and finding out if there are big gaps there, because right now, the way it tends to work is it's after the fact,

right people file their taxes, and then a certain number of those taxes tax reports are picked to be looked over in more careful detail, and it's only if they start to detect a pretty uh significant pattern that they'll look at any individual's taxes specifically, unless you're part of some political controversy which we won't get into, but this big data thing would allow you to take a look at a much larger scale and focus in on particular problems,

as opposed to just hoping that the sheet of reports that you just pulled from the printer includes people who are not paying their fair share. So I've got a question. Yeah, now that we're talking about the government using big data to predict near duells, and uh, I think I see where this is going. Yeah, y'all seen that movie Minority Report, documentary Minority Report. Well, okay, so I want to explain a little bit. In that movie, they've got a they've

got a division of law enforcement called pre crime. Where they are now in the movie, it's kind of they've got these like psychic prelugs. But let's just say replace the psychics with really, really really powerful computers, right that look at trends pattern com make extremely accurate predictions about what's about to happen. I can definitely foresee a future where it might not be all that impossible for computers to predict when somebody is very likely to commit a crime?

Could I can? I can tell you. Let me give you a little more, a little more insight. From my perspective, I don't know that we're going to get to a point where we're going to be able to predict when a specific individual is likely to commit a crime. We can definitely get a little more probabilistic, you know, sit there and say, what is the probability of any person at any given time to commit a crime? Um, there

are some things that we can say. For example, there are law enforcement agencies there that are now using big data in order to predict crime trends. So not a specific person not saying, you know, yeah yeah ne'er dowell, Johnny today is gonna knock over the liquor store. They are not doing that. What they are doing is saying, looking at this big data, I'm seeing this trend where this particular part of town tends to be a target

for vandalization. In burglary, let's say those those are two crimes that often tons of factors, you know, based on weather or right. Apparently things like burglaries, um kind of go in rashes. Yeah. That's another thing is that if a place is hit by burglars, then there is there tends to be a increased risk of the same thing

happening in and in the sanginal area. Yeah, so if there's a successful burglary attempt in one particular home, for example, other homes in that neighborhood could be also um prone to being hit by burglars. So that's one example that

law enforced we can use. They can use it as a reactionary thing, saying all right, well, because we know this, we should end up increasing patrols in this area for the time being so that we can discourage any other crime or catch the criminals before they're able to hit another another house or another business. Another thing is that for crimes like burglary and vandalism, those are crimes that generally go down when you increase patrols. They they are

considered low intensity but high frequency crimes. So if you were to adjust patrols so that there is a more frequent patrol of police through that area, you reduce the

likelihood of those crimes being committed. And by using big data and and and really analyzing where these crimes are taking place within a city, you can redraw patrol routes so that police are taking the most efficient patrol they can, so they're not having to patrol an area that's way larger than what they're capable of doing in a in a given shift, and you also will hit the areas that are most likely to be targeted and help reduce crime.

That way, you're preventing it from happening. So you're not going out and arresting someone for a crime they haven't committed yet. That's not the same thing at all. But you can help attack those sort of crimes, things like murder much less you know, much less prone to any sort of pattern that you can predict. It's that's something that's a high intensity but low frequency crime as opposed

to low intensity, high frequency like vandalism and burglary. So they don't tend to take that kind of crime into consideration when they're looking at this big data in this sense, other than to uh perhaps say that, you know, this particular area of town needs to have a stronger police presence in order to help can back to what we were talking about with sample size, Yeah, and uh, Like in Santa Cruz, California, police use this approach to and identify homes that were more likely to be hit by

a burglar so they could redraw their patrol roots to take that into consideration and prevent that from happening. Now, when we do talk about criminals and the likelihood of someone to to commit a particular crime, there is some statistical evidence to suggest that people who are who have committed a crime are more likely to commit another crime than someone who has never committed a crime like that,

there's like a fort recidivism rate. But part of that is due to the way that we handle criminals and how we try to reintroduce criminals to society. So it may not be that people just have this statistical likelihood of committing a crime again once they've already done. So some of its institutionalize, right, It's a social construct, not a personal propensity. Right, So therefore that wouldn't it wouldn't be the issue what the cause was. It would just

be like that, you see it. Well, no, there's an issue about what the cause was, because if you can treat the cause, then you remove the I'm saying that criminals matter, and Joe, you're just throwing them away. I think you know, you know, I'm talking about what the cause was. Wouldn't affect how how how well? Just like I was, I was teasing Joe, but we were very

clearly talking about two different things of the same problem. Yeah, And and what is scary about all of this is the thought that someone could say, well, all of all of this is is reactive in looking for this kind of crime, and why can't we be proactive and well

and getting into that scary minority. And that's and and I think, I mean, I'm not saying that we'll never get to a point where we where where statistical models won't give, at least again, a probabilistic approach of how likely is person A to commit a crime versus person B. And you take everything into account, and you compare that against all the information you've ever gathered and come up with a probability that's probably gonna happen at some point.

But I don't think that we're ever going to act on that. I'm just saying that, I think if a computer can predict that Jonathan Strickland is likely to buy a horsehead mask, it can also probably predict that Jonathan Strickland is more likely than the average person to Robert Jimmy Johns. But what was what horrifying how accurate you are? But there's a Jimmy Johns with then walking distance of

this office. What we should always keep in mind is even if computers are that good, we shouldn't ever let that prejudice our approach to Jonathan Strickland, because he may very well not buy a horsehead mask, and he may very well not Robert Jimmy Johns. That's true. That was my point. Okay. So yeah, So while while Minority Report definitely had this sort of scary science fictionary approach to you know, uh, stopping people arresting people for crimes they

had not yet committed, but we're going to come it. Uh. And and they, you know, they had the benefit of having psychics who are apparently infallible, except they're not. When you watch the movie spoiler. Yeah for a movie that's that old. I'm sorry anyone who's listened to this. Yeah. So anyway, Uh, I don't think we're ever gonna I don't think we're ever going to get to a point where big day is when pre crime comes up. Vote no right to your representative, say no to pre crime things.

You got to vote no one. Vote no one pre crime and vote no on giving artificial intelligence the right to vote. Those are the two things you have to make sure deals. Yeah, those are two big strikes, Lauren, can you tell us something happy? I can? Well, okay, So, so part of what part of what is scary about all this data is that it's really hard for us to understand what's out there, what we're generating, what it's

being used for, and what all that looks like. I mean, because you know, like we were talking about like at a certain point, we're like, oh, sure, a quadrille, what's what's the number. It's a lot, Yeah, it's it's a it's a one with fifteen zeros. And that's a lot of zeros. And and there are a group of data artists out there. Are you giggling at more than a dozen zeros? That is more than a dozen zero? Let's thank you. Let's not let Joe talk anymore. Take the

X away from Joe. I haven't been touching it. Please

please go on. Um, they're there are a group of data artists out there who are working to to put all of this into some kind of meaningful and and also culturally meaningful unit that that we can process and u And there there's there's one particular fellow by the name of Jared Thorpe who used to be the data artist in residence at the New York Times and has as of I think December or January of UM gone off and found did the Office for Creative Research as

as it's being called a company of his UM and and he he posits that that this data art is going to help people understand what all of this data

means and what it's being used for. UM and he's got some really interesting just personal projects that he did a TED talk that's that's pretty pretty terrific UM or you can you can see, you know, he's he's taken Twitter data from from people saying good morning and putting it into this kind of gorgeous bouncy map of of just of just tracking when people are waking up and saying good morning to Twitter. He also shows what time they say good morning based upon the color of the

block that appears. So if they say good morning earlier, it's a green block, and the later they say good morning, it goes into goes into the reds. And so he also could show trends that way, like around the world, showing trends of when people would say good morning and uh in general, Let's say the West Coast wakes up at around eleven am and the East coast we're early risers, not only because the sun gets to us first, but because the west coast is sleeping in. It's all those

actors who say good morning at two pm. You're an actor, that's true, I am, but I get up at you know, five in the morning. The trick is you don't say good morning, just a grump coffee everyone. That's pretty much meet hate everyone? That what what when am I not

tweeting that I hate everyone? So so that's an interesting example of data visualization right right, and and and that's that's what they're working for, is that visualization of getting something down to a graphic scale where we can go like, oh, that's still a way too huge for me to comprehend, but at least it looks kind of pretty and I get it now. I can totally see that how that would help people understand what data means. H um he was talking about in one UM one talk that he

gave it pop tech. I believe about about how people have been saying that data is the new oil, and and how kind of grandiose and lovely that sounds. For a second, because people are thinking like oil, oil is money, money, is good. But but but it's how how terrifying that is in a certain way because because oil for for you know, a very specific example has been a resource that has been so misused and is so poisonous and terrible global instability and war and and all of this.

But and that that you know, similarly, this data could be used or misused rather for um, you know, not very good purposes like like we were talking about. But if we you know, if if we use these kind of resources, and by resources, I mean people who are processing this, um too, get away from all of the capitalism that the capitalism is terrible. But but but using this data for the greater common good, for some things like predicting influenza outbreaks and being able to respond quickly

before it becomes a pandemic. I mean, clearly you're talking about benefiting potentially millions of people. We've seen flu outbreaks affect millions of people, and if you're able to respond fast enough so that you could contain that, then that would be an obvious, you know, benefit to everybody. So yeah, that's and that's just one example. There's some that are more like, well, this makes my life easier. The traffic

stuff for example. But even even in the bigger scheme, if you talk about traffic, that seems kind of trivial. You know, all it means that I don't have to spend you know, extra time sitting in traffic. That also means you're spending less time running a gasoline power at engine I mean, unless you have an electric vehicle or whatever. But and the stress levels which relate to your to your heart rate and health, um, the your productivity at work.

If you could get everyone in Atlanta to work in half an hour less than they currently spend on the road, I mean, we would probably just be looking at pictures of cats on the internet anyway. But um, but either way, yeah, yeah, no, I agree entirely. So there are and you know, I like the I like the artistic vision of showing this as a way to demonstrate this is just one way of looking at the information. And uh, and you know, the ways that you've heard of are just the tip

of the iceberg. We haven't even really explored the full extent of what we can use this data for. And in some cases it may be truly transformative. We won't have to necessarily reinvent or or invent brand new technology to make the world a better place. We may have all the tools already, it's just in that information we have to feagure. Yeah, and then part of that is getting people interested in this field and creating a culture

around it. Yeah. Yeah, agreed. Well that's awesome. I mean it's I've only seen one of those, uh those examples. I saw the good morning example for Twitter. It was a spinning globe and all the little uh pop ups of showing where people had said good morning. Um. It also made me feel better about my tweets because I don't. I don't tend to say good morning. No, I don't know. I also don't LaVar Burton does. Is LaVar Burton not good enough for you? No, LaVar Burton is good enough

for me. Um. You know, I I like to take He was data. That was terrible. His data's best friend was about to say, uh, what was this just the worst next generation? Well, yeah, okay, So LaVar Burton can say good morning, that's fantastic. I don't. I don't have enough followers to say good morning. I do occasionally quote half of a song lyric here's a there's a shock just to see how many people who follow me know

what I'm quoting and see. Yeah, so if you know the end of the sky is blue and all the grass is green, my heart's as full as a baked potato. You let me know, all right. Well, I think that wraps up our discussion about applications of big data and and what we're using it for. And again, it's just kind of a hint at what big data will be

used for. And while yes, there are certainly examples of how companies, governments could abuse big data in a way that are legitimately scary, there are also some truly amazing uses that could be very beneficial. So I don't think we should shy away from it because of the uh, the possibility of things being used in a scary way. We just need to be aware of it and be and make sure that we don't go down that pathway because the benefits are too great for us just to ignore. Well,

of course, it can go either way. I mean, it's elemental, it's knowledge and know. Yeah, it's a tool, and you know a tool is it's going to be used the way the person who's using the tool wants to use it, So like a drill, so a third so so not nice you really carried it home there, We're just going to I'm just gonna end here, guys. If you have suggestions for future episode topics or you want to tell me the end of the song lyric I quoted rite

us let's know. Are you know? Just as FW Thinking at discovery dot com or go to f W thinking dot com. Check out our blogs, check out the podcasts, check out the videos. We've got some really fun ones up there. I think you guys will really like it, and we'll talk to you again really soon. For more on this topic in the future of technology, is it Forward Thinking dot Com Brought to you by Toyota. Let's go Places,