WEBVTT - Big Data and You 0:00:01.000 --> 0:00:04.320 From UFOs, two, ghosts and government cover ups. History is 0:00:04.400 --> 0:00:08.000 riddled with unexplained events. You can turn back now or 0:00:08.080 --> 0:00:17.599 learn the stuff they don't want you to now. Hello, 0:00:17.680 --> 0:00:19.520 welcome back to the show. My name is Matt and 0:00:19.720 --> 0:00:22.439 i'm Ben. We're here with our super producer Noel, which 0:00:22.480 --> 0:00:25.720 makes this stuff they moge want you to know. Close 0:00:25.800 --> 0:00:28.840 snug quite there. You're right, I said something wrong. I 0:00:28.880 --> 0:00:32.040 can't figure out what it was. Yeah, we're pretty excited 0:00:32.159 --> 0:00:35.599 about this new gadget or I guess new to us 0:00:35.720 --> 0:00:37.839 gadget that's here in the studio to day, Matt, do 0:00:37.840 --> 0:00:39.639 you want to tell everybody a little bit about it? 0:00:39.640 --> 0:00:43.000 It's just a couple of old school synthesizers from the 0:00:43.040 --> 0:00:46.440 seventies and eighties, and hopefully no one's going to be 0:00:46.840 --> 0:00:49.680 bringing those in the mix. Maybe on this show, maybe not. 0:00:49.840 --> 0:00:51.600 I don't know. I don't know. He looks like it 0:00:51.680 --> 0:00:55.400 could go either way. So let's start today's episode with 0:00:55.560 --> 0:00:58.600 an anecdote that you may have heard if you watched 0:00:58.640 --> 0:01:03.960 our video series earlier this week two thousand twelve. Minnesota, 0:01:04.240 --> 0:01:07.760 and this family starts getting ads from Target. You know 0:01:07.880 --> 0:01:11.560 every people get ads from Target, right, You're used to it. 0:01:11.560 --> 0:01:15.880 It happens, but this time something's different. They've received ads before, 0:01:15.880 --> 0:01:18.559 but this time something's off. The ads are for stuff 0:01:18.600 --> 0:01:23.039 like diapers, strollers and baby lotion and stuff. You get 0:01:23.080 --> 0:01:27.120 the idea, like for someone who's expecting lovely thing to be. 0:01:27.240 --> 0:01:31.080 But there's a catch here. Nobody at that house is expecting. 0:01:31.520 --> 0:01:35.000 So the dad is livid, and the especially about the 0:01:35.000 --> 0:01:38.880 ads addressed specifically to his teenage daughter. That's the whole 0:01:38.959 --> 0:01:43.600 point to his high school aged daughter. Yeah, yep. And 0:01:43.920 --> 0:01:47.560 so he goes to Target in person on in some 0:01:47.600 --> 0:01:52.440 beast mode WTF kind of stuff trending towards w w E. 0:01:52.520 --> 0:01:58.080 Possibly when he goes back, has a chat with his daughter, 0:01:58.600 --> 0:02:03.160 comes back to Target and says, oh, yeah, she's expecting 0:02:03.200 --> 0:02:07.680 in August. Whoops on all accounts. This comes to us 0:02:07.880 --> 0:02:10.960 through a New York Times piece that was published on 0:02:11.120 --> 0:02:17.400 February nineteenth, two thousand twelve, about how obsessively companies track 0:02:17.520 --> 0:02:21.519 your shopping habits and every little piece of information they 0:02:21.560 --> 0:02:24.960 can about you to hopefully make you more likely to 0:02:25.120 --> 0:02:29.880 buy something. And the guy who was interviewed, the statistics 0:02:29.880 --> 0:02:32.520 man working for Target interviewed in this New York Times 0:02:32.639 --> 0:02:38.600 article was shut down after he mentioned this because what 0:02:38.960 --> 0:02:41.960 we're talking about is something that many many companies do 0:02:42.160 --> 0:02:46.720 but is incredibly controversial, and that is big data. Yeah, 0:02:46.800 --> 0:02:50.960 it's something that it's not necessarily something they don't want 0:02:51.000 --> 0:02:52.400 you to know, but they don't want you to know 0:02:52.560 --> 0:02:55.959 about it, Like it's kind of a known thing now 0:02:56.080 --> 0:02:58.919 that you are being tracked in all these different ways 0:02:58.960 --> 0:03:02.239 and looked at. But this is something that we feel 0:03:02.320 --> 0:03:06.600 you should know more about, right, Yeah, it's something that 0:03:07.120 --> 0:03:09.960 it's it's strange that you say it's not necessarily something 0:03:10.000 --> 0:03:12.880 they don't want you to know, because they certainly don't 0:03:12.960 --> 0:03:16.360 want other people to know the techniques used to gather 0:03:16.440 --> 0:03:19.520 this information. Absolutely, they might not want you to know 0:03:19.639 --> 0:03:23.280 some of the things that we can't tell you because 0:03:23.400 --> 0:03:28.639 we don't know, right, because the data sets themselves are important, 0:03:28.639 --> 0:03:31.440 But what maybe even more important and even more secretive 0:03:31.440 --> 0:03:35.000 would be the techniques used to parson analyze this information 0:03:35.360 --> 0:03:40.000 and even where they're getting some of this data from. Ah. Yes, yeah, 0:03:40.000 --> 0:03:42.640 So what what is big data? If we have to 0:03:42.680 --> 0:03:45.120 define it, it's not like it's not the same thing 0:03:45.160 --> 0:03:52.600 as uh, big agribusiness or big bell or something big pomegranate. Right, No, 0:03:53.040 --> 0:03:57.640 big data is just as in definition, any extremely large 0:03:57.760 --> 0:04:01.200 data set or data set so that can be analyzed 0:04:01.480 --> 0:04:04.240 usually now in the modern day computationally is the only 0:04:04.280 --> 0:04:07.480 way to even get your wrap your head around it 0:04:08.080 --> 0:04:11.920 um to reveal patterns and trends all kinds of associations, 0:04:12.080 --> 0:04:16.159 especially those relating to human behavior and or the interactions 0:04:16.200 --> 0:04:20.120 between humans. So, for instance, this big data would not 0:04:20.279 --> 0:04:23.840 be the GPS of one person. It would be the 0:04:23.920 --> 0:04:28.680 GPS of a city, or people who all work at 0:04:28.720 --> 0:04:33.760 a large company or yeah, one one GPS companies data. 0:04:34.240 --> 0:04:36.640 Yeah or yeah, But I love the idea of the 0:04:36.680 --> 0:04:40.800 GPS information of one like certain area and just all 0:04:40.839 --> 0:04:43.960 of that data stacked vertically together from everyone that's been 0:04:44.000 --> 0:04:47.280 in that area. Right. There are different ways to parse this, uh. 0:04:47.360 --> 0:04:53.000 This big data usually include sets that are so large 0:04:53.120 --> 0:04:57.960 there beyond the ability of typical software tools Like you couldn't, 0:04:58.000 --> 0:05:02.160 for instance, just take uh, some data set that is 0:05:02.200 --> 0:05:05.000 big data size and put it in an Excel spreadsheet 0:05:05.120 --> 0:05:07.760 on a Google travel. No, there's no way, and if 0:05:07.800 --> 0:05:09.960 you did, it would be the largest file in the 0:05:10.040 --> 0:05:14.039 history of XL. So when we look at the range 0:05:14.040 --> 0:05:15.960 of this, we also know that the size of data 0:05:16.000 --> 0:05:20.839 is a constantly moving target. Because it seems that there's 0:05:20.880 --> 0:05:23.760 just more and more and more out there. There's a 0:05:23.760 --> 0:05:26.320 really weird statistic here somewhere that I think is like 0:05:26.760 --> 0:05:29.960 nine percent of all the data that exists was made 0:05:29.960 --> 0:05:33.840 in the last twenty or ten years or something. Oh. Absolutely, 0:05:33.880 --> 0:05:35.839 it's exponential. And that's one of the things we're gonna 0:05:35.880 --> 0:05:40.000 talk about here. Just the number of devices that collect 0:05:40.160 --> 0:05:43.440 data or that you can collect data upon, and just 0:05:43.640 --> 0:05:47.960 everything that's connected to the Internet nowadays. It's insane. And again, 0:05:48.120 --> 0:05:51.119 it doesn't go away once. Once you have a data set, 0:05:51.480 --> 0:05:54.440 it's not like it's irrelevant. You're still going to need 0:05:54.480 --> 0:05:57.919 that maybe for future, you know, whatever endeavor is that 0:05:57.960 --> 0:06:01.160 you're going to do as a big business. Right for gamers, uh, 0:06:01.400 --> 0:06:04.880 this would be sort of the equivalent of an inventory 0:06:05.000 --> 0:06:09.839 that doesn't register weight. How role playing games turn and 0:06:09.880 --> 0:06:11.600 you're more familiar with this, and I am, how role 0:06:11.640 --> 0:06:15.640 playing games turn so many people into orders. We are 0:06:16.400 --> 0:06:20.760 data orders as well, like the NSA clearly is. Uh. So, 0:06:20.880 --> 0:06:24.719 what we'll do is will walk through some history of 0:06:24.800 --> 0:06:28.039 big data, and then we'll also talk about some of 0:06:28.080 --> 0:06:30.719 the controversy surrounding it, the ways it could be used, 0:06:30.720 --> 0:06:36.840 some of the conspiracies theoretical and factual about this practice. 0:06:37.400 --> 0:06:40.880 So one way that one way there's a group called 0:06:40.920 --> 0:06:44.279 Meta Group or Gardner. Uh. They they have an analyst 0:06:44.279 --> 0:06:47.839 there named Doug Laney, and he said, one great way 0:06:47.880 --> 0:06:51.760 to measure big data would be in the three vs. 0:06:51.800 --> 0:06:55.400 That would be increasing volume, uh, just how much data 0:06:55.480 --> 0:06:58.360 you have, Uh, the velocity of the data, the speed 0:06:58.400 --> 0:07:01.320 of the in and out right, and variety that type 0:07:01.360 --> 0:07:03.640 of stuff, where is it coming from? So not just 0:07:03.720 --> 0:07:07.599 all homogeneous GPS records, but also like what something else 0:07:07.640 --> 0:07:10.200 people would collect on another set of data that's really 0:07:10.520 --> 0:07:14.000 interesting to look at when when combined with GPS data, 0:07:14.040 --> 0:07:18.280 our phone records, that's always a fun thing. Um. And 0:07:18.320 --> 0:07:22.000 these these three vs that you're mentioning, this is now 0:07:22.280 --> 0:07:25.800 an industry standard the way the whole industry looks at 0:07:25.800 --> 0:07:29.000 big data with what is it? Volume? Velocity? What was 0:07:29.040 --> 0:07:33.440 the third one? That's volume, velocity and variety. So uh, 0:07:33.560 --> 0:07:36.800 GPS and phone records would be something that that helps 0:07:36.800 --> 0:07:40.480 you flesh out a virtual persona, but it comes from 0:07:40.600 --> 0:07:43.640 the same device probably for most people. True. So another 0:07:43.680 --> 0:07:45.520 thing that would be more very would be stuff like 0:07:45.920 --> 0:07:50.280 medical records, stuff like recent purchases on your financial records, 0:07:50.280 --> 0:07:53.760 so you could part it down to uh people who 0:07:54.000 --> 0:07:55.560 let's get a little bit dark with it. You could 0:07:55.600 --> 0:07:58.640 parse it down to uh some We have these four 0:07:58.920 --> 0:08:01.360 sets of data out this person, right, So we know 0:08:01.520 --> 0:08:06.600 that once every week or something they go to a 0:08:06.840 --> 0:08:10.320 clinic right and specializes in some kind of treatment, right. 0:08:10.560 --> 0:08:14.520 And then we know that the medical records that they 0:08:14.600 --> 0:08:18.440 have some kind of uh, debilitating condition. And then we 0:08:18.480 --> 0:08:23.080 see that one of their recent purchases is a skydiving suit. 0:08:23.880 --> 0:08:27.200 So we have built this picture with just the very 0:08:27.240 --> 0:08:31.440 little information about someone who probably has a terminal disease 0:08:31.720 --> 0:08:34.240 and wants to skydive because it was on their bucket list. 0:08:34.320 --> 0:08:36.560 Now we know exactly the kind of things to sell 0:08:36.600 --> 0:08:38.640 to this person, and we know exactly the kind of 0:08:38.679 --> 0:08:41.880 things to sell, which is frightening. Uh. So this is 0:08:41.960 --> 0:08:44.120 I mean, this is an example that you and I 0:08:44.280 --> 0:08:47.160 just made up now, right, Matt, this is not we 0:08:47.480 --> 0:08:50.040 don't know any more and that this has happened to 0:08:50.800 --> 0:08:56.319 so this is a new information age. It's tough to 0:08:56.400 --> 0:09:02.720 stress how quickly personal privacy has eroded, right, um, and 0:09:02.760 --> 0:09:08.400 we the people of the information age, have committed um, 0:09:08.760 --> 0:09:12.480 have have lost our privacy due to acts of negligence, 0:09:12.559 --> 0:09:17.960 not due to aggressively rooting for this loss of privacy. 0:09:18.080 --> 0:09:21.880 It's just who reads terms and conditions, right, Yeah, And 0:09:22.600 --> 0:09:25.719 we're all very happy about some of the things that 0:09:25.840 --> 0:09:29.320 providing this type of data gives to us. The GPS 0:09:29.360 --> 0:09:31.839 thing alone, I think is a massive I mean a 0:09:31.920 --> 0:09:35.160 lot of people would I don't. I can't speak for everyone, 0:09:35.200 --> 0:09:37.599 but for myself to have the ability to use the 0:09:37.640 --> 0:09:40.480 GPS when I'm out on the road somewhere instead of 0:09:40.520 --> 0:09:42.920 having to look at a physical map is one of 0:09:42.920 --> 0:09:46.280 those things where sometimes I just go, you know what, 0:09:46.360 --> 0:09:49.240 screw it, turn my GPS on. I need it, right, 0:09:49.440 --> 0:09:53.320 And even if they know that this, were highly aware 0:09:53.840 --> 0:09:57.640 that this is being tracked. But but there's also there's 0:09:57.679 --> 0:10:03.800 such a a sort of narcissism in paranoia, this idea 0:10:03.920 --> 0:10:07.840 that yes, of course they care where I get my pickles. 0:10:08.640 --> 0:10:11.440 Of course, of course someone does. And don't get me wrong, 0:10:11.440 --> 0:10:14.160 I mean sure grocery stores for sure do. Every time 0:10:14.200 --> 0:10:18.199 you swipe a loyalty card, you are generating more information 0:10:18.280 --> 0:10:22.160 for them to uh target ads toward you, which we'll 0:10:22.160 --> 0:10:25.160 talk about whether or not that is, you know, morally 0:10:25.200 --> 0:10:28.439 wrong or ethically sticky or whatever or if you would, 0:10:28.559 --> 0:10:31.640 would someone want that? Like, are there people out there 0:10:31.679 --> 0:10:33.840 that want Kroger to know exactly what the order so 0:10:33.880 --> 0:10:35.840 they can just say, hey, here's all the things that 0:10:35.880 --> 0:10:38.320 you want and hear the coupons right there? And there 0:10:38.360 --> 0:10:41.320 are some people I would definitely take some coupons for things. 0:10:41.640 --> 0:10:44.040 But also, this is okay, this is so unrelated. This 0:10:44.120 --> 0:10:48.160 is a side conspiracy here. And Matt, uh, So you 0:10:48.280 --> 0:10:52.880 and I are just old enough to remember before these 0:10:52.920 --> 0:10:55.720 loyalty cards came out everywhere, right, Yeah, when I was 0:10:55.760 --> 0:10:59.160 in college, they didn't exist. So the way that they 0:10:59.200 --> 0:11:02.360 were instant you did was that for for all the 0:11:02.440 --> 0:11:04.320 young guns out there, the way that these cards were 0:11:04.320 --> 0:11:08.720 instituted was at first they gave you discounts on things. Right. 0:11:09.440 --> 0:11:13.240 But one of the conspiracies I've heard is that this 0:11:13.800 --> 0:11:18.520 artificial price shenaniganting if I can make up the word, uh, 0:11:18.800 --> 0:11:23.120 ultimately became something where having the card didn't really get 0:11:23.120 --> 0:11:26.520 you a discount. It just got you the the regular price. 0:11:26.600 --> 0:11:29.200 The regular price, yes, and the money that they're making 0:11:29.240 --> 0:11:33.680 on top of selling your information to third parties. CBS 0:11:33.760 --> 0:11:36.840 is like yeah, thanks, yeah, it's and and so you're 0:11:36.960 --> 0:11:40.280 you're being penalized at least, this theory goes for not 0:11:40.440 --> 0:11:45.679 participating in this program. And that's because again, the overwhelming 0:11:45.760 --> 0:11:48.960 majority of companies, uh, don't just do this. They do 0:11:49.040 --> 0:11:51.960 this with relish and it is profitable. But it is 0:11:52.000 --> 0:11:54.439 a very very old idea. Is that I have to 0:11:54.480 --> 0:11:57.079 say this, here's the worst news about that whole situation. 0:11:57.120 --> 0:12:00.000 What's that even if you go somewhere and shop. I'm 0:12:00.040 --> 0:12:01.680 just gonna use an example, but I'm aware of like 0:12:01.760 --> 0:12:04.800 publics that I don't have a loyalty card for and 0:12:04.880 --> 0:12:06.560 I don't know that you can get one. Maybe you 0:12:06.559 --> 0:12:09.960 can now, but and you you try and save money there, 0:12:09.960 --> 0:12:13.160 if you pay with a card of any type, then 0:12:13.400 --> 0:12:16.000 that stuff is being tracked, maybe by a separate company, 0:12:16.080 --> 0:12:18.800 a third party, or maybe just your credit card company. 0:12:18.840 --> 0:12:21.960 But the same thing is happening. And we'll see a 0:12:22.000 --> 0:12:25.160 couple of different reasons that this happens, you know, or 0:12:25.400 --> 0:12:29.360 or a couple of different motivations for this collection. But 0:12:29.480 --> 0:12:32.200 let's walk through the history first. Okay, so this is 0:12:32.240 --> 0:12:35.560 not a new thing. Trying to track data, trying to 0:12:35.800 --> 0:12:40.079 understand numbers about things that are happening in the world 0:12:40.080 --> 0:12:42.920 around you. Goes back seven thousand years to something that 0:12:42.960 --> 0:12:48.679 we've mentioned before in Mesopotamia during the birth of agriculture, 0:12:49.720 --> 0:12:51.880 when you had to keep track of seeds, you had 0:12:51.880 --> 0:12:55.480 to keep track of crops and soil, just everything you 0:12:55.520 --> 0:12:58.760 needed to know data about the ground and the plants 0:12:58.840 --> 0:13:01.880 that you're trying to put in there. So yeah, So, 0:13:02.000 --> 0:13:06.719 for example, there's an accounting system that goes in to 0:13:07.040 --> 0:13:12.240 monitor the growth of a I don't know, uh, an 0:13:12.240 --> 0:13:16.880 olive sure, an olive tree, right, and so, um, you 0:13:16.920 --> 0:13:20.680 know some farmer named John Stamos, a very common name 0:13:20.679 --> 0:13:23.679 in Mesopotamia at the time. I imagine, Uh, this farmer, 0:13:23.760 --> 0:13:27.719 John Stamos has uh, a bunch of olives, and they 0:13:27.760 --> 0:13:30.320 have to have a way to track year over year 0:13:30.480 --> 0:13:33.840 the performance of that crop. Right. So this is when 0:13:33.840 --> 0:13:36.839 they begin saying, Okay, you know John Stamos has X 0:13:36.880 --> 0:13:42.000 amount of trees. They yield y amount of olives each year. 0:13:42.120 --> 0:13:44.200 This is what we can expect. This is what we 0:13:44.240 --> 0:13:47.000 can expect, this is what we can bet against, this 0:13:47.120 --> 0:13:50.360 is what we can sell an advance before it's made. 0:13:51.440 --> 0:13:55.000 Then you get into we're gonna jump forward pretty far here, 0:13:55.120 --> 0:13:59.320 all the way to sixteen sixty three. Nothing else happened, yeah, zero, 0:14:00.440 --> 0:14:05.000 nothing between seven thousand years ago. Let's just say improvements 0:14:05.440 --> 0:14:10.200 are being in incrementally, you know, are happening up until 0:14:10.240 --> 0:14:12.560 this point. But then on the next big change, the 0:14:12.600 --> 0:14:16.200 sixteen sixty three when John Grant, I think that's how 0:14:16.240 --> 0:14:20.560 you correctly spell it. Grant very British. Uh. He recorded 0:14:20.560 --> 0:14:25.720 an examined information about mortality rates because the bubonic plague 0:14:25.840 --> 0:14:29.880 was just ravaging just the entire area at the time, 0:14:30.120 --> 0:14:32.400 and he decided he wanted to know more information about, 0:14:32.440 --> 0:14:34.960 like how what is happening here, exactly how many people 0:14:35.000 --> 0:14:37.400 are getting affected? Why are they getting affected? Let's get 0:14:37.440 --> 0:14:41.280 information and we can start solving this. Yeah, and that 0:14:41.320 --> 0:14:44.160 sounds that sounds sensible. It's the least you could do, 0:14:44.320 --> 0:14:47.600 right we I don't know, have we ever talked on 0:14:47.600 --> 0:14:50.760 this show about just how profoundly the plague or the 0:14:50.920 --> 0:14:54.040 series of things known as the plague change the world. No, 0:14:54.400 --> 0:14:57.720 I've listened to a little too much stuff you missed 0:14:57.720 --> 0:15:00.720 in history class about it. So the information, Saan sometimes 0:15:00.800 --> 0:15:03.080 gets muddled between what we've talked about and what I've 0:15:03.080 --> 0:15:07.520 just heard. These plagues have played such a profound role 0:15:07.800 --> 0:15:12.360 in uh, the global evolution of the human species. It's 0:15:12.360 --> 0:15:14.080 just crazy it's the kind of stuff you would want 0:15:14.080 --> 0:15:18.440 to keep track on. So this guy, uh, John Grant becomes, 0:15:19.040 --> 0:15:22.160 uh the father of statistics, or he's considered that because 0:15:22.200 --> 0:15:25.600 he does the first statistical data analysis that we have 0:15:25.640 --> 0:15:28.440 recordings of, uh, and he has a book about it 0:15:28.520 --> 0:15:32.720 called Natural and Political Observations Made upon the Bills of Mortality. 0:15:33.160 --> 0:15:35.360 Just kind of a dry name, right, but it's it's 0:15:35.400 --> 0:15:40.600 not a feel good subject, and people continue to work 0:15:40.680 --> 0:15:44.400 off this statistical analysis. So let's fast forward to the 0:15:44.440 --> 0:15:47.560 twenty century. Okay, so we're going to fast forward all 0:15:47.600 --> 0:15:52.760 the way to eighteen eight seven. This is when the 0:15:52.840 --> 0:15:57.560 modern the age of modern data, modern data is when 0:15:57.600 --> 0:16:01.840 it is born. So this gentleman named Herman Hollerith invented 0:16:01.880 --> 0:16:06.280 a computing machine that could read these holes punched holes 0:16:06.400 --> 0:16:11.160 in cards that paper cards in order to organize census data. 0:16:11.560 --> 0:16:14.120 And this is a huge change because you have to 0:16:14.160 --> 0:16:17.640 imagine just collecting data, of going door to door getting information, 0:16:17.640 --> 0:16:21.200 then trying to compile that just with humans in rooms. 0:16:21.880 --> 0:16:25.160 That it was taking so they would do one one 0:16:25.200 --> 0:16:28.480 census every ten years, I believe at the time, and 0:16:28.600 --> 0:16:31.280 it was taking almost nine years before you would get 0:16:31.480 --> 0:16:34.720 the results from the census of the previous the previous census. 0:16:35.400 --> 0:16:39.320 So it was almost I don't not worthless, but it 0:16:39.440 --> 0:16:42.200 was just felt like they were running backwards almost right. Yeah, 0:16:42.360 --> 0:16:44.560 they had they had a data set that would be 0:16:44.600 --> 0:16:47.720 when it was finally complete, useful for a little less 0:16:47.720 --> 0:16:51.880 than a year. Yeah, exactly so. Uh. The first data 0:16:52.000 --> 0:16:56.600 processing machine appeared in nineteen forty three. This was, of course, 0:16:56.640 --> 0:16:59.880 it came out of war. A lot of technological in 0:17:00.080 --> 0:17:02.960 vations come out of war. Uh. And it was meant 0:17:03.000 --> 0:17:07.720 to decipher codes from the Nationalist Socialists or the Nazis. Uh. 0:17:07.760 --> 0:17:11.040 This thing was named Colossus, which is a pretty cool name. 0:17:11.440 --> 0:17:14.320 And they would intercept messages. They would feed things to 0:17:14.400 --> 0:17:17.359 Colossus and would search for patterns in these characters. And 0:17:17.720 --> 0:17:21.360 it worked pretty quickly. Yeah. It would go five thousand 0:17:21.440 --> 0:17:24.960 characters per second, which is huge. It reduced the time 0:17:25.040 --> 0:17:28.919 from weeks two hours. Let's stroll through some other stuff here, 0:17:29.000 --> 0:17:30.639 just kind of laundry listed so you can get to 0:17:30.680 --> 0:17:34.040 the good stuff. Uh. Nineteen fifty two, everybody's favorite, the 0:17:34.119 --> 0:17:37.000 n s A, the National Security Agency is created, and 0:17:37.040 --> 0:17:41.800 within ten years they have more than twelve thousand cryptologists 0:17:41.840 --> 0:17:46.880 on contract. That's huge. Then you've got. In nineteen sixty five, 0:17:46.960 --> 0:17:50.800 the US government builds the first data center which can 0:17:50.880 --> 0:17:55.399 store seven hundred and forty two million tax returns and 0:17:55.440 --> 0:17:58.480 also a hundred and seventy five million sets of fingerprints. 0:17:58.880 --> 0:18:02.600 Now this is pretty interesting here because this is this 0:18:02.680 --> 0:18:06.200 is something that was recorded on the magnetic tape and 0:18:06.400 --> 0:18:10.160 computer tape. You may, I don't know if anybody listening 0:18:10.200 --> 0:18:13.160 would know what that is. Hopefully maybe you've heard of 0:18:13.200 --> 0:18:17.080 this magnetic tape. My father is a controller controlling account 0:18:17.119 --> 0:18:19.760 and they have a like a newer version of this 0:18:19.800 --> 0:18:22.480 magnetic tape, but it's still all of their stuff is 0:18:22.520 --> 0:18:25.200 backed up to this magnetic tape because it's so well, 0:18:25.200 --> 0:18:29.720 it's supposedly so reliable. Well, also, people will recognize the 0:18:29.760 --> 0:18:33.280 magnetic tape if you've never seen it before. Um, every 0:18:33.320 --> 0:18:36.960 time you see an old computer with reels on it, 0:18:37.080 --> 0:18:40.840 that's magnetic tape. In Captain America the Winter Soldier, the 0:18:40.840 --> 0:18:44.280 sequel to the First Captain America, Uh, there is a 0:18:44.440 --> 0:18:46.560 scene which I won't spoil for you if you haven't 0:18:46.560 --> 0:18:48.439 gotten around the scene it yet, but there is a 0:18:48.440 --> 0:18:53.760 scene which involves a gigantic computer and that is magnetic tape. Awesome, Okay, 0:18:53.840 --> 0:18:57.720 good reference. Now we understand. But here here's the thing. Though, 0:18:57.720 --> 0:19:00.840 This whole project was scrapped because of fears of quote 0:19:01.000 --> 0:19:04.200 big brother right, being a little bit too big brotherish 0:19:04.280 --> 0:19:11.679 little Orwellian. So this, however, changed everything because people were thinking, 0:19:11.760 --> 0:19:16.399 what if we centralize, um, the location of data, you know, 0:19:16.520 --> 0:19:22.160 no more paper, just electronically store it. A British guy, 0:19:22.240 --> 0:19:24.320 so you may have heard of tim Berners Lee of 0:19:25.280 --> 0:19:28.840 Invince what will go on to become the World Wide Web? 0:19:29.800 --> 0:19:34.520 And with this foom, we're going like gangbusters because people 0:19:34.520 --> 0:19:37.840 are able to generate massive amounts of information, much more 0:19:37.960 --> 0:19:42.240 so than anybody could plausibly read. And when it's connected 0:19:42.280 --> 0:19:45.960 up to this uh interweb, if you will, it can 0:19:46.040 --> 0:19:49.760 be it can be sent somewhere else, right. And then 0:19:49.880 --> 0:19:52.160 if you do have a data center, doesn't matter where 0:19:52.160 --> 0:19:54.760 the data is collected, you can send it directly over 0:19:54.800 --> 0:19:58.400 there almost immediately. Yeah, it's it's bizarre when we think 0:19:58.440 --> 0:20:03.280 about that, and especially we think about um, how just 0:20:03.400 --> 0:20:05.920 let's have a John Henry moment, and can you compare 0:20:06.000 --> 0:20:08.680 matt the the ability of a supercomputer to a person. 0:20:09.160 --> 0:20:17.359 Oh God, can I m hmm. Let's say okay, let's 0:20:17.400 --> 0:20:21.720 say in the first supercomputer. Let's use that one as 0:20:21.920 --> 0:20:25.800 our example. It could do as much work in one 0:20:25.880 --> 0:20:30.760 second then a single human being operating a calculator could 0:20:30.840 --> 0:20:36.960 do in thirty thousand years. Thirty thousand years. And that's 0:20:37.000 --> 0:20:39.480 not even the twenty one century, ladies and gentlemen. Now 0:20:39.560 --> 0:20:41.920 we are at the modern age. And two thousand five, 0:20:41.960 --> 0:20:45.399 a guy from O'Reilly media coined the term big data 0:20:45.520 --> 0:20:48.800 for the first time. Uh and this was, you know, 0:20:48.840 --> 0:20:53.040 a successor to a another less fortunate buzzword, which was 0:20:53.280 --> 0:20:58.760 web two point oh yeah, yeah, the new coke of 0:20:58.840 --> 0:21:03.560 web words. But yeah, so this this idea here is 0:21:03.600 --> 0:21:06.359 a little bit more bad of the idea of data 0:21:06.400 --> 0:21:09.919 set that is just so massive and complex and interwoven 0:21:10.080 --> 0:21:14.720 that you can't use the traditional business intelligence tools to 0:21:14.800 --> 0:21:17.919 figure out what's going on. Right And oh five is 0:21:17.960 --> 0:21:20.680 also the year that Hoddup was created by Yahoo, which 0:21:20.840 --> 0:21:23.560 was built on the back of Google's map produce. And 0:21:23.600 --> 0:21:28.359 these are just softwares that can basically take data from 0:21:28.440 --> 0:21:32.080 using a bunch of different computers to crunched numbers like 0:21:33.160 --> 0:21:37.000 and huge amounts um And it was the goal to 0:21:37.040 --> 0:21:40.920 index the entire or the entire worldwide Web. That's why 0:21:40.960 --> 0:21:44.040 these things were created and uh, it's the open source 0:21:44.040 --> 0:21:46.560 to dupe. It was used by a lot of organizations 0:21:46.600 --> 0:21:50.639 to crunch through data. That just is it's almost in 0:21:50.760 --> 0:21:53.119 quantifiable how huge it is. Right, and this is not 0:21:53.240 --> 0:21:56.040 just a private industry thing, of course, And the line 0:21:56.160 --> 0:21:58.480 is blurry. We seem to talk about these two events 0:21:58.480 --> 0:22:02.119 in isolation, as if the n s A using phone 0:22:02.160 --> 0:22:06.600 records and social media contacts. Who do you know that 0:22:06.680 --> 0:22:10.280 knows who? That knows who on Facebook? Right on the list? 0:22:10.400 --> 0:22:13.400 Now you're on the list right. Uh, they're they're not 0:22:13.480 --> 0:22:19.080 just using that, Uh and target or another private organization, um, 0:22:19.720 --> 0:22:23.119 a data broker. There are companies that just broker data. Uh. 0:22:23.320 --> 0:22:27.320 These do not exist in a vacuum. There's interplay between them, 0:22:27.520 --> 0:22:34.080 and uh, they're increasingly merging to do just some amazing things. 0:22:34.080 --> 0:22:38.320 We're getting very very good at seeing the present as 0:22:38.400 --> 0:22:41.120 never before. And other governments are involved in this as well. 0:22:41.160 --> 0:22:44.560 In two thousand nine, the Indian government did something just 0:22:46.160 --> 0:22:49.640 ambitious is like the most reasonable word for this. They 0:22:49.640 --> 0:22:53.280 decided to take an Irish scan, fingerprint and photo of 0:22:53.880 --> 0:22:59.000 everyone in India, every single person in India. There are 0:22:59.200 --> 0:23:05.479 so many people, Yes, can you imagine listener Uh, if 0:23:05.920 --> 0:23:08.160 the government came to you and said, Okay, we're going 0:23:08.200 --> 0:23:11.480 to need we're gonna need an Irish skin of fingerprint, 0:23:11.520 --> 0:23:13.480 and we're also going to need a photograph, a really 0:23:13.600 --> 0:23:18.159 nicely framed photograph with good contrast. We're gonna put it 0:23:18.240 --> 0:23:21.080 in of you and everyone in your family put it 0:23:21.080 --> 0:23:23.479 into this database. Don't worry. We're gonna make sure it's secure, 0:23:23.920 --> 0:23:25.919 and we're not going to use it for anything but 0:23:26.080 --> 0:23:29.720 for good things. Right, Yeah, and that's one point to 0:23:30.400 --> 0:23:34.520 billion people to bill. Yeah, this is the largest biometric 0:23:34.600 --> 0:23:40.439 database in the world. So there's a great thing that 0:23:40.600 --> 0:23:44.080 Eric Schmidt from Google also said, right, just another sense 0:23:44.080 --> 0:23:48.040 of perspective here, yeah, he he stated at the Techonomy 0:23:48.280 --> 0:23:53.120 conference in Lake Tahoe. He he stated, quote, there were 0:23:53.160 --> 0:23:57.359 five exabytes of information created by the entire world between 0:23:57.359 --> 0:24:01.359 the dawn of civilization and two thousand three. Now that 0:24:01.480 --> 0:24:07.160 same amount is created every two days. Boom. Take take 0:24:07.240 --> 0:24:09.399 that history of the universe. I can't wait till the 0:24:09.440 --> 0:24:13.200 aliens land and say uh that they're like, Wow, these 0:24:13.240 --> 0:24:17.040 guys figured out how to make uh pizza into burritos 0:24:17.040 --> 0:24:21.560 and there's a blog about it. It's sort of like 0:24:21.800 --> 0:24:26.919 when you know, when people modern times find ancient Greek 0:24:27.119 --> 0:24:31.480 or Roman or African ruins from these empires of bygone days. 0:24:31.640 --> 0:24:34.640 And there's always some jerk like hundreds of years ago 0:24:34.880 --> 0:24:38.879 who wrote like Tim was here, yeah and true, like 0:24:38.920 --> 0:24:42.000 dick butt from Reddit. They're just like, what is a 0:24:42.119 --> 0:24:45.520 precursor right there? Right? So there's but there's so much 0:24:45.600 --> 0:24:48.520 information being made and you know, I'm being a little 0:24:48.560 --> 0:24:51.159 crass here, but the point I'm hoping to make is 0:24:51.240 --> 0:24:55.760