Speaker 1: Brought to you by Toyota. Let's go places. Welcome to
Speaker 1: Forward Thinking, either everyone, and welcome to Forward Thinking, the
Speaker 1: podcast that looks at the future and says, what's the story,
Speaker 1: Morning Glory, what's the word hummingbird? I'm Jonathan Strickland, I'm
Speaker 1: laurenco and I'm Joe McCormick, and we have jazz hands
Speaker 1: and today you know it totally does not translate to radio.
Speaker 1: But anyway, we're doing Yeah, and we were. Actually I wasn't,
Speaker 1: but you two were honesty and podcasting. That's true. So
Speaker 1: today we wanted to talk about the concept of big
Speaker 1: data or big data, which is not a gigantic character
Speaker 1: from Star Trek the Next Generation. Unfortunately, No, that would
Speaker 1: have been both interesting and terrifying. But that is not
Speaker 1: the case. Actually, I guess you could still argue that
Speaker 1: big data is both interesting and to some people terrifying. Yeah,
Speaker 1: it's it's so what is data? I've got a couple
Speaker 1: of official definitions, and then Joe, I think you have
Speaker 1: your own definition, So let me let me go through
Speaker 1: these quote unquote official definitions. These are from IBM, and
Speaker 1: IBM is one of those companies that has a lot
Speaker 1: invested in big data in general and big data management
Speaker 1: and so in a paper called Demystifying Big Data. Here's
Speaker 1: one of the definitions, which is big data is a
Speaker 1: phenomenon defined by the rapid acceleration and the expanding volume
Speaker 1: of high velocity, complex, and diverse types of data. Big
Speaker 1: data is often defined along three dimensions, volume, velocity, and variety.
Speaker 1: And then the other definition is big data as a
Speaker 1: term that describes large volumes of high velocity, complex and
Speaker 1: variable data that require advanced techniques and technologies to enable
Speaker 1: to capture, storage, distribution, management, and analysis of the information. So, Joe,
Speaker 1: what's your what's your definition? That was a lot of words. Yeah,
Speaker 1: it was, uh, well it just seems to me and
Speaker 1: I'm no expert here, but big what's the difference between
Speaker 1: just a lot of data to and then big data.
Speaker 1: That's a good question because we've had a lot of
Speaker 1: data before, but suddenly there's sort of this new paradigm
Speaker 1: where we have to think about, oh, it's big data.
Speaker 1: It's a separate thing. It's not just a matter of degrees.
Speaker 1: And I think it's the point at which our intuition
Speaker 1: kicks in and tells us something weird is going on, right,
Speaker 1: there's so much here to handle that I can't even
Speaker 1: imagine handling it. Yeah, it's it's it's when you suddenly realize, oh,
Speaker 1: hold onto your butts. The phrase was actually entered into
Speaker 1: the Oxford English Dictionary just this quarter, like this month.
Speaker 1: As of June, they entered it into their quarterly online
Speaker 1: update and and there there there's a definition that is
Speaker 1: very much like the one that Jonathan read off, but
Speaker 1: slightly more succinct. Um, well, it's the O E. D Well.
Speaker 1: They also include several several examples of it of its
Speaker 1: use um the original going back to when a social
Speaker 1: historian um by name of C. Tilly referenced it. But
Speaker 1: in terms of meaning being obscured when you are in
Speaker 1: the presence of an increasingly complex system of of of information. Right,
Speaker 1: so it has something to do with it's hard for
Speaker 1: us to grasp intuitively. Right. Well, well, yeah, it's kind
Speaker 1: of like, if you want to boil it down to
Speaker 1: a cliche, it's the whole not being able to see
Speaker 1: the forest for the trees. Like you're able to see
Speaker 1: the stuff that's immediately around you, but when you're trying
Speaker 1: to get a bigger picture, your perspective is blocked by
Speaker 1: the fact that there's just so much there. So if
Speaker 1: you can't get a good grasp on the big picture,
Speaker 1: so exactly how many trees are we talking about here, Well,
Speaker 1: let's let's boil that down. Let's boil that down to
Speaker 1: talking about how we measure data in the computer world.
Speaker 1: And for anyone who has any background in computers at all,
Speaker 1: this is probably going to seem super basic to you.
Speaker 1: But it's important to have the building blocks there for
Speaker 1: us to understand the enormity of big data. Right, if
Speaker 1: you go look on the internet, it seems like a
Speaker 1: lot of times people confuse terms like data and facts
Speaker 1: and information like they just use them interchangeably. Data and
Speaker 1: information are you know, of course that they are synonyms.
Speaker 1: But when we're talking about data in the terms of computation,
Speaker 1: and then we're talking about bites, right, So a bite
Speaker 1: is eight bits, and one bite can represent one character.
Speaker 1: So when I'm talking about character, I'm talking about like
Speaker 1: a letter or a number or a symbol. I'm not
Speaker 1: talking about Jean Valjean from La Misseraba. That's totally different
Speaker 1: kind of character. So eight bits can be one character,
Speaker 1: So it takes about ten or bytes, So eighty bits
Speaker 1: total ten bytes to make up about you know, one
Speaker 1: word or so. Uh So that's your basic unit of data.
Speaker 1: So if we look at kill a bite, that's we
Speaker 1: we say it's a thousand bytes. Technically it's one thousand,
Speaker 1: twenty four bytes. And in counting, yeah, this this gets
Speaker 1: a little, this gets a little complex as we go,
Speaker 1: high are up, so I will be rounding down to
Speaker 1: the nearest. Uh yeah, because otherwise, uh we'll be spending
Speaker 1: the entire podcast just listening to me read out an
Speaker 1: incredibly long number. I'll give an example of that when
Speaker 1: I get to it, but I'll skip over most of
Speaker 1: them anyway. So a kilobyte will rough says roughly a
Speaker 1: thousand bites. So if you were to type out a
Speaker 1: page of text, that would be about two kilobytes of information.
Speaker 1: Now that's if you're just typing text, not like images
Speaker 1: or anything else. But that would be about two kilobytes. Now,
Speaker 1: if you had a low res photo, that's probably around
Speaker 1: a hundred kilobytes, maybe fewer, depending upon the resolution. I
Speaker 1: mean that you know some things that if you save
Speaker 1: it for the web, it can be between like thirty
Speaker 1: and a hundred kilobytes or so. Uh, the next step
Speaker 1: up is megabyte. So that's technically one million, forty eight thousand,
Speaker 1: five D seventy six bites, but we'll usually just say
Speaker 1: one million. And that's the last I'm going to do
Speaker 1: of the specific numbers. Uh, so high res photo could
Speaker 1: be at east two megabytes. Five megabytes is enough to
Speaker 1: hold the complete works of Shakespeare. So now wait a minute,
Speaker 1: would that be plain text or formatted documents? It would
Speaker 1: be essentially plain text? Yeah, plain text would be about
Speaker 1: five megabytes Shakespeare. Yeah, now if you want to if
Speaker 1: you want there's there's all these bears pursuing you in
Speaker 1: Winter's Tale, so you got to fill that out. Uh.
Speaker 1: The a CD ROM, do you guys remember those? There's
Speaker 1: still they still exist. I have one in my computer
Speaker 1: in front of us, an optical drive. They round, they
Speaker 1: were uh you know, along one side, but if you
Speaker 1: looked at them in profile, they were flat. That's what
Speaker 1: you used to play missed right, Yes it was in fact,
Speaker 1: but a CD ROM could hold between about six fifty
Speaker 1: and nine hundred megabytes. And we're talking about the twelve
Speaker 1: centimeter discs, not the eight centimeter discs, because they had
Speaker 1: many discs as well. It wasn't as popular here in
Speaker 1: the United States, but in Asia they were very popular.
Speaker 1: Then you have gigabyte that ends up being the billion
Speaker 1: mark of bytes. Again, I'm just simplifying here about one
Speaker 1: gigabyte can hold a broadcast quality movie. By the way,
Speaker 1: I remember when I had a computer that had a
Speaker 1: I think three gigabyte hard drive, and that was so huge. Yeah.
Speaker 1: I remember when I got a two and fifty six
Speaker 1: megabyte hard drive and I thought that there's no way
Speaker 1: anyone could fill up that budget space. And look at
Speaker 1: what I know. A twenty gigabyte drive could hold the
Speaker 1: high fidelity recordings of the entire works of Beethoven. A
Speaker 1: fifty gigabyte hard drive is equivalent to a floor of
Speaker 1: books in a typical library. The next step up is terabyte,
Speaker 1: which is one trillion bytes. That two terabytes would be
Speaker 1: equivalent to an academic research library, and ten terabytes would
Speaker 1: be equivalent to all of the printed collection at the
Speaker 1: United States Library of Congress, just the printed materials. It's
Speaker 1: amazing how much cheaper the storage has been come over
Speaker 1: the years. What does it cost to go buy like
Speaker 1: a one tero byte external hard drate? It all depends
Speaker 1: on where you go, but you're talking around a hundred
Speaker 1: couple hundred dollars at most for most places. And and
Speaker 1: the thing is that you know, the the technology has
Speaker 1: improved over time and the manufacturing processes have improved over time,
Speaker 1: which has brought the price down over time. But we're
Speaker 1: not done yet. We gotta go back. So if you
Speaker 1: that was one trillion bites, let's say you one of
Speaker 1: the levels, let's say you want to quadrup Well, the
Speaker 1: numbers we're going to be talking about are way bigger
Speaker 1: than well, they don't know, we haven't said it yet.
Speaker 1: So next is A is one quadrillion bites. That's a
Speaker 1: that's a pedo byte and uh, to pedal bytes would
Speaker 1: be all US academic research libraries combined. Two hundred peda
Speaker 1: bytes would be all printed material everywhere. Uh. If you
Speaker 1: went up to one quintillion bites, I don't know how
Speaker 1: many zeros that is, it's a lot. Uh, that's exabyte
Speaker 1: and five exabytes would be enough to contain all the
Speaker 1: words ever spoken by human beings. If you were to
Speaker 1: break down all the words we've ever spoken into bites,
Speaker 1: it would fit within five exabytes. Did we say whose
Speaker 1: estimates these are? Oh? Well, these are actually estimates that
Speaker 1: are up all over the place there. You know. IBM
Speaker 1: actually cites these as their their benchmarks as well. Yeah,
Speaker 1: and then the next two levels up you want to
Speaker 1: go up. So we've done quintillions, So sextillion would be
Speaker 1: zeta bite and I guess septilian would be YadA bite.
Speaker 1: So anyway, or yakta YadA, not yoda, not yoda, not
Speaker 1: yakta not yakta YadA. So so anyway, those those are
Speaker 1: the scales exabite. We get to the point where it's
Speaker 1: all the words we've ever spoken, right, So that gives
Speaker 1: you the idea of of the basics of bites and
Speaker 1: how much information is equivalent to you know, various real
Speaker 1: world examples. So let's talk about the amount of information
Speaker 1: that we create on a daily basis. Soil for example,
Speaker 1: of corey DIABM, we create about twelve terabytes of tweets
Speaker 1: in one day. That's twelve trillion bytes of messages at
Speaker 1: a hundred forty characters or fewer tweets alone. And Twitter
Speaker 1: isn't even by far the most popular social networking No
Speaker 1: is not is not the most popular at all. And
Speaker 1: yet that's not even rich text isn't. That's just plain text.
Speaker 1: It's plain texts short messaging service really is what it is.
Speaker 1: So so remember too, terabytes is equivalent to a single
Speaker 1: academic research library. So you've got the equivalent of six
Speaker 1: research libraries academic research libraries, just in tweets alone. Now,
Speaker 1: I'm not saying that you're going to be able to
Speaker 1: research your next uh thesis only using Twitter. It's not
Speaker 1: necessarily useful data that's out there, but that's how much,
Speaker 1: you know, once you put it together. Lots of people
Speaker 1: are are tracking trends and keywords and and you know,
Speaker 1: sure all kinds of things. There are lots of useful
Speaker 1: Twitter to predict the stock market and earthquakes, and they're
Speaker 1: using Twitter for all sorts of stuff. Yeah, consumer behavior,
Speaker 1: Not that we're endorsing those ideas, by the way, but
Speaker 1: going back to the kind of data that we actually
Speaker 1: are producing on a daily basis. Back in two thousand twelve,
Speaker 1: when Facebook still had just under one billion registered users,
Speaker 1: they were collecting According to Facebook, they did an earnings
Speaker 1: call where they said they were collecting about five hundred
Speaker 1: terabytes per day from users. So that's all the stuff
Speaker 1: that everyone is doing on Facebook, whether they are posting
Speaker 1: a status update or liking a page or sharing a link.
Speaker 1: All of that was folded into this number. But five
Speaker 1: hundred terabytes every day. So then you've got YouTube. You guys,
Speaker 1: of course have heard the famous UH stat Now it
Speaker 1: is one hundred hours of video that's uploaded every minute
Speaker 1: between the time we shot that video and by the
Speaker 1: time we but between the time we wrote it and
Speaker 1: the time we shot, really time we shot in the
Speaker 1: time it published exactly. Yeah, it went from seventy two.
Speaker 1: Technically it had been growing all that time, but Google
Speaker 1: gave the official announcement. Yeah, so one hundred hours of
Speaker 1: YouTube footage is upload every minute. So that means that
Speaker 1: in a single day you get about a hundred and
Speaker 1: forty four thousand hours of video on YouTube added every day.
Speaker 1: That's sick. It's a little it's a little stomach churning actually. Um.
Speaker 1: And so if you were to look at all the
Speaker 1: data that we are creating, and this is beyond social media,
Speaker 1: we're talking about all the information being created not just
Speaker 1: by human beings, but by things like sensors that are
Speaker 1: connected to computer systems and are sending that data in
Speaker 1: So things like whether uh sensors or traffic sensors, the
Speaker 1: cameras we've talked about in the past. All of this
Speaker 1: stuff combined generates about two point five quintillion bytes of
Speaker 1: data also known as XA bytes every single day. Well,
Speaker 1: that makes me curious, does more data come from human
Speaker 1: entry or from other machines? Right now, humans are actually
Speaker 1: generating most of that data. Eight percent of it, in fact,
Speaker 1: is coming from unstructured information, which includes things like email, video, blogs, uh,
Speaker 1: social media, call center conversations. All of this goes into
Speaker 1: that that data that's going to change. Probably if you
Speaker 1: remember our Internet of Things episode, we talked a lot
Speaker 1: about how we're going to be living in this world
Speaker 1: where our environments are going to be constantly collecting Yeah,
Speaker 1: the more devices are collecting information about us and yeah,
Speaker 1: yeah it Well, it'll it'll get to a point where
Speaker 1: we'll start to see that number probably uh fluctuate quite
Speaker 1: a bit. We'll see that percentage drop for human produced
Speaker 1: data versus you know, automated sensors. So two point five
Speaker 1: quintillion bytes of data produced every day, that means every
Speaker 1: two days we produce as much information as all the
Speaker 1: words we've ever spoken. So that's kind of incredible and also,
Speaker 1: according to ibm UH, in the last two years, we
Speaker 1: have produced of all the world's data, meaning that everything
Speaker 1: prior to those two years represents of all the data. Ever,
Speaker 1: it's a it's a pretty crazy, crazily curved graph as
Speaker 1: of So, so, you know, years ago at this point,
Speaker 1: Eric Schmidt of Google said that we were creating as
Speaker 1: much info every two days as as we had from
Speaker 1: the dawn of human kind up through two thousand three. Yeah,
Speaker 1: so in one point we created one point eight zeta
Speaker 1: bytes of information globally UH in that year, and that
Speaker 1: amount is expected to double every year. So that means
Speaker 1: one point a zeta bites. If you want to know, like, okay,
Speaker 1: well what does that mean to me? Like, how how
Speaker 1: can I conceptualize this amount of information? That's equivalent to
Speaker 1: two hundred billion two our HD movies? And if you
Speaker 1: wanted to watch those two hundred billion HD movies and
Speaker 1: just sit down and have a marathon, it would take
Speaker 1: you forty seven million years to do it, no bathroom breaks. Yeah,
Speaker 1: that that those these these numbers don't even make any
Speaker 1: sense to me at that point. It's just you know,
Speaker 1: and that's and that's the that's the key, right that
Speaker 1: that's why we call it big data, because they are
Speaker 1: such huge numbers that when you think about you're like,
Speaker 1: what what can I even do with all this information?
Speaker 1: How can I make use of it? But that's exactly
Speaker 1: the thing, right, We're going to a point where it's
Speaker 1: not so much that you're going to do something with it,
Speaker 1: but the machines are going to do something right And
Speaker 1: and there's some very creative processes that people have come
Speaker 1: up with that break this down into more manageable problems
Speaker 1: that machines can handle. Well, I shouldn't say that you're
Speaker 1: not going to use it for sure. I mean, this
Speaker 1: kind of thing is probably really useful to people who
Speaker 1: are involved in, say marketing. Well, it's useful for marketing,
Speaker 1: but it's also and we'll do yeah, well, we'll do
Speaker 1: a full episode on some of the applications of using data. Yeah,
Speaker 1: that will be our next episode that we recorded. Me
Speaker 1: But but yeah, there here's an example that all of
Speaker 1: us in this room could use UH with big data.
Speaker 1: Now we would not be actually doing the analyzing ourselves,
Speaker 1: but we would benefit from the UH, the the the
Speaker 1: actual work, and that would be traffic, real time traffic
Speaker 1: reports So if you are using some sort of GPS
Speaker 1: that allows you to get incoming traffic information that's being
Speaker 1: gathered by various means, and there are different ways of
Speaker 1: doing it depending upon what system you're using, then you
Speaker 1: are essentially benefiting from the from the analysis of big
Speaker 1: data because it's taking all this information about the actual
Speaker 1: environment around you that's gathered from multiple sources and helping
Speaker 1: you route the most efficient way. Right, does it dynamic routing?
Speaker 1: Which is really a cool thing and and and obvious benefit.
Speaker 1: But that's just one application. So how do you end
Speaker 1: up navigating this much information? Like what what's the what's
Speaker 1: the magic key to it? And there are actually a
Speaker 1: couple different ways. I don't want to I don't want
Speaker 1: to throw my co host under the bus because I
Speaker 1: specifically look this stuff up. And so I ask the question.
Speaker 1: They're like, I, uh, it's a lot throw a dart.
Speaker 1: I don't know. Um, you use computers, that's good, Joe, supercomputers.
Speaker 1: Perhaps you could use supercomputers. You could also use grid computing,
Speaker 1: which is where you end up using a lot of
Speaker 1: computers to work on a single problem. So you guys
Speaker 1: have probably heard the term parallel processing, parallel parallel processing
Speaker 1: is is an idea where you are able to take
Speaker 1: certain kinds of computer problems where you can divide up
Speaker 1: the problem into different UH sections, maybe even subsections of data,
Speaker 1: and then assign each of those parts to a different processor.
Speaker 1: And this this could happen within a computer, or you
Speaker 1: could be talking about spreading it out lots of different computers.
Speaker 1: So if you have a computer that has multiple cores,
Speaker 1: for example a multi core processor, each core could be
Speaker 1: taking part of this problem and working on it separately.
Speaker 1: And really what you have is you have you essentially
Speaker 1: have one unit acting as the director, and the director's
Speaker 1: job is to take the problem and to divide it
Speaker 1: up into manageable chunks, and then to parcel that out
Speaker 1: to all the other elements of the system, whether it's
Speaker 1: other computers or other processors or other cores. Their job
Speaker 1: is to work on that particular part of the problem
Speaker 1: and then send the results back to the master. The
Speaker 1: master then takes all the results and has a collective
Speaker 1: result as as the final product, and that's where you
Speaker 1: get the answer that you're looking for UH. It's called
Speaker 1: often this this approach is called a map reduce framework.
Speaker 1: You're mapping out the problem and then routing it out
Speaker 1: and then you reduce the problem that way, when all
Speaker 1: the uh and when all the answers are sent back
Speaker 1: to you, you you reduce all those answers into one answer.
Speaker 1: So that's the whole process for taking generally a huge
Speaker 1: problem and making it manageable. Now, the key to all
Speaker 1: of this, and a lot of people and companies that
Speaker 1: specialize in big data will tell everyone this is that
Speaker 1: you can't just do anything with this information. What you
Speaker 1: have to do is decide what is it that you
Speaker 1: want to do with the information, specific thing are we
Speaker 1: looking for, right, and then you build a system that
Speaker 1: lets you get that from derive that from all the
Speaker 1: data you have. So it's not like you just look
Speaker 1: at it. Yeah, you can't just look at this big
Speaker 1: ball of zeros and wines and then just magically draw
Speaker 1: out the information you need. What you do and or
Speaker 1: you just sit there and you look at and say,
Speaker 1: you know, what can we learn from this? If you
Speaker 1: look for a specific kind of pattern, like if you're
Speaker 1: looking for a needle in a haystack, you're you're looking
Speaker 1: for something shiny, for example, and and you know, if
Speaker 1: you if you just go like, yeah, if you just say, well,
Speaker 1: I'm looking for something kind of pointy and short. Then
Speaker 1: that's not What is interesting is that when you get
Speaker 1: information on this scale, this huge amount of information, you
Speaker 1: can actually start to recognize patterns that otherwise would have
Speaker 1: been completely Yeah, you would never have been able to Again,
Speaker 1: it's the forest for the trees thing. You would never
Speaker 1: have been able to have seen the forest because you
Speaker 1: were right there in the middle of all those trees.
Speaker 1: So the same sort of thing, you'd be able to
Speaker 1: see these big patterns that happen. And that's where especially
Speaker 1: things like marketing ends up being a big deal, because
Speaker 1: you can see things like tendencies for customers to behave
Speaker 1: in a certain way, and if you want them to
Speaker 1: behave of a particular way, you can start to focus
Speaker 1: on things that kind of guide them in that direction.
Speaker 1: But I've even seen bizarre representations a thing online that
Speaker 1: was the strange corn ucopia shape and it was just
Speaker 1: labeled like the geometry of big data. Yeah, it's it's
Speaker 1: a little like again, when we're talking about something so
Speaker 1: huge that it's difficult for us to get a mental
Speaker 1: grasp on it, trying to find a representation of that
Speaker 1: that makes sense. To us is something of an uphill
Speaker 1: battle it. You know, it's a lot of people have tried,
Speaker 1: but it's really difficult to make this and make it
Speaker 1: understandable in a way that doesn't just blow out the
Speaker 1: scale immediately, where you know, you have the manageable amount
Speaker 1: of data and then the spike just goes all the
Speaker 1: way up through the top of the graph and you
Speaker 1: can't see the topic. That's so it's really big. Other
Speaker 1: ways to handle all that data, there's also uh the
Speaker 1: approach of doing just real time analytics and streaming of data.
Speaker 1: In this case, this would be kind of like traffic
Speaker 1: example I gave earlier. So with traffic, you have all
Speaker 1: these sensors gathering data, and then you have uh that
Speaker 1: analysis and streaming of the data happening immediately, and then
Speaker 1: you get the results. UH. In this case, you don't
Speaker 1: have to worry so much about storing lots of data
Speaker 1: because it doesn't really matter if there was a slow
Speaker 1: spot on the highway two hours ago. What matters is
Speaker 1: what's going on right now, So you don't have to
Speaker 1: worry about building these huge data centers to store all
Speaker 1: that information. You just have to build a system that's
Speaker 1: large enough to handle incoming information and give outgoing information.
Speaker 1: So you have to have a good input output basis. Uh,
Speaker 1: that's what's important in those types of systems. Now, in
Speaker 1: the other types of systems where you are collecting and
Speaker 1: analyzing enormous amounts of information, you have to have a
Speaker 1: place for that information to live, and that's where storage
Speaker 1: comes into play. And that's where we see these enormous
Speaker 1: data centers things that buildings are specifically made to house
Speaker 1: data servers. So it's if you were to walk into
Speaker 1: one of these places, it essentially would look like a
Speaker 1: huge warehouse or maybe even like an airline hangar. I mean,
Speaker 1: these buildings can be enormous and they're filled with shelves
Speaker 1: of servers. They usually have massive HVAC systems. Sometimes they
Speaker 1: ideally they are because of course, the warmer things get,
Speaker 1: the more poorly technology can perform. To an extent. You
Speaker 1: don't want to super cool everything because then it can
Speaker 1: have its own problems, but you do want it to
Speaker 1: maintain a decent operable temperature. So you might have even
Speaker 1: a water cooling system and not just air cooling. They
Speaker 1: have to have sort of a distributed redundancy to don't
Speaker 1: like because if one machine dies and with that many machines,
Speaker 1: you know, just every so often several machines are going
Speaker 1: to die. You you can't lose something, right. So for example, Google,
Speaker 1: which is that's a great example, because Google has lots
Speaker 1: of out of centers, uh, and that Google uses famously.
Speaker 1: They use fairly inexpensive servers in the grand scheme of things.
Speaker 1: They're not buying the top of the line, fresh off
Speaker 1: the manufacturing plant servers. They want things that are plentiful
Speaker 1: and easy to replace. Yeah, they're going for efficiency, not
Speaker 1: high power. Yeah, they don't. They don't need each server
Speaker 1: to be able to handle the workload of three other servers.
Speaker 1: They want things that are going to be reliable and
Speaker 1: if it does break down, make it easy to switch
Speaker 1: it out with something else. But on their system, they
Speaker 1: do have lots of redundancy. And it's this idea that
Speaker 1: you know, stuff breaks, machines go down, power goes out.
Speaker 1: With that many it's just statistically guaranteed exactly you know
Speaker 1: what's going to happen. So the way you protect against
Speaker 1: it is that you build extra you have extra machines
Speaker 1: involved there so that some of them have a little
Speaker 1: bit of information from like if you have servers A
Speaker 1: through Z, server DEM might have a little bit of
Speaker 1: information from server A, and then maybe even server you know,
Speaker 1: J has a little bit from server A. So it's
Speaker 1: the ideas that you've spread it out so that if
Speaker 1: anyone server goes down, you still have access to that information,
Speaker 1: so that there's no interruption in service. Uh. Now there
Speaker 1: are cases of servers that have gone down where that
Speaker 1: was the only really source of that information and that
Speaker 1: has been a spectacular failure. You mean my data is
Speaker 1: not safe. Well, I'm not going to say that, Joe.
Speaker 1: That's not Let's not spread fear, uncertainty, in doubt. That's
Speaker 1: not what this podcast is about. Now, It's not that.
Speaker 1: It's just that there have been times in the past
Speaker 1: where it became clear like this is the way to go,
Speaker 1: The redundancy way is the way to go, and I
Speaker 1: would I would say that, you know, I can't imagine
Speaker 1: any operation of the size that would involve big data
Speaker 1: would not also have redundancy plans there. And then of
Speaker 1: course that does mean that you have to have even
Speaker 1: more machines than what you would require at minimum, and
Speaker 1: that requirement is constantly going up. But as we're generating
Speaker 1: more and more data every day. So then the question
Speaker 1: becomes data management. You know, how long do you keep
Speaker 1: that information? At what point do you you know, do
Speaker 1: you ever wipe uh a drive so that you can
Speaker 1: you know, fill it up again? Or it all depends
Speaker 1: on what your your services and what the purpose of
Speaker 1: it is. But uh, yeah, I mean that's it's it's
Speaker 1: kind of crazy, like how much how much infrastructure needs
Speaker 1: to exist just so that these zeros and ones have
Speaker 1: a home? Well, so if we extrapolate that outward, that
Speaker 1: leads me to what seems like kind of a weird
Speaker 1: philosophical question almost Uh, is there a limit to the
Speaker 1: kind of data we can process? And ultimately, if you
Speaker 1: say no, there's no real physical limit, there's no necessary limit.
Speaker 1: Is it possible to represent the entire universe, all of
Speaker 1: reality as data? Or is there something about the universe
Speaker 1: that can't ever be reduced to information? I am going
Speaker 1: to tackle your question in multiple parts. Part the first, uh,
Speaker 1: is there a limit? I I hesitate to ever say
Speaker 1: that there is a limit in the sense that there
Speaker 1: there's always more innovation that allows us to do bigger
Speaker 1: and better things. But I will say there is a
Speaker 1: limit to the amount of energy that is in the universe, right, yeah, yeah,
Speaker 1: I was just thinking about like, well, they're say physical constants,
Speaker 1: and there's like the speed of light and stuff like that,
Speaker 1: but we're talking about things that ever represent a barrier. Well,
Speaker 1: I don't think we found any kind of physical law
Speaker 1: that states that once you get to this amount of data,
Speaker 1: there's no amount of parsing that you can do to
Speaker 1: make it useful. I don't think, well, certainly, we haven't
Speaker 1: encountered that yet. I don't think it's I don't think
Speaker 1: that's possible, simply because as we get more and more data,
Speaker 1: we're also building more and more powerful machines that can
Speaker 1: handle larger amounts of data. And if we're able to
Speaker 1: break down those problems into smaller bits anyway, then really
Speaker 1: the limitation we're the limiting factor we're looking at here
Speaker 1: is energy, not computering power. So although at the current moment,
Speaker 1: some computer scientists are concerned about the amount of data
Speaker 1: that we're crunching versus there they're saying that the amount
Speaker 1: of data that we're creating is um fast, outstripping More's
Speaker 1: law in terms of how fast processors are going. Sure,
Speaker 1: there will there will be bottlenecks. I mean, there will
Speaker 1: obviously be bald next. But if you're looking at truly philosophical,
Speaker 1: idealistic approach, you're essentially saying that eventually you could create
Speaker 1: more data than you could possibly process, only because you
Speaker 1: don't have enough there's not enough energy in the universe
Speaker 1: to run all the processors you would need to handle
Speaker 1: that much data. There's that or like I had the
Speaker 1: crazy absurd idea. I know this is silly, but like
Speaker 1: you've got so much data that the server farm is
Speaker 1: so big that the pieces of information are too far
Speaker 1: apart to communicate with each other efficiently. Like physically, I
Speaker 1: can see what you're saying. So you're saying like like
Speaker 1: we had we had yeah, if we had planets, it
Speaker 1: all depends on on Yeah, I can see what you're saying.
Speaker 1: So let's say that Let's say that we're filling up
Speaker 1: space with servers. This again is a very philosophical kind
Speaker 1: of you know, thought experiment approach. But let's say we're
Speaker 1: filling up fifty years out where he filled up space
Speaker 1: with with with computer servers, and you were just packing
Speaker 1: space with the servers so that you could process more
Speaker 1: and more data. You could get to all right, enough
Speaker 1: of the comedy. I'm trying to make a point here.
Speaker 1: You can't. You could get to a point in that
Speaker 1: in that theoretically where you've got a server that literally
Speaker 1: is light years away, could be physically next to billions
Speaker 1: of other servers, but it's away from where. Yeah, then
Speaker 1: you're talking about you're limited by the speed of communication.
Speaker 1: Not it's again not really the p assessing limitation. It's
Speaker 1: the speed of light that's limiting you. But but you'd
Speaker 1: be able to do it, it would just take time. Um,
Speaker 1: your other question, could all of the universe, all reality itself,
Speaker 1: be broken down into data? I don't know, because first
Speaker 1: of all, we're only able to observe part of the universe,
Speaker 1: and we don't know how much of the rest of
Speaker 1: it there is. Uh. But assuming that we could, that
Speaker 1: would mean that all right, let's let's assume that it
Speaker 1: is possible to break down all of reality into zeros
Speaker 1: and one sure, sure, in that twenty fifty years, we
Speaker 1: have figured out what dark matter is and we have
Speaker 1: observed all of it. Right, We've got we've got to
Speaker 1: we've got it down. Our fingers on the pulse of
Speaker 1: the universe. And we know what makes it tick, and
Speaker 1: then we can actually create a simulation of that because
Speaker 1: we know that, we can break down the universe itself
Speaker 1: into what you know, into data, make that transition. That
Speaker 1: would then raise the argument of well, if we could
Speaker 1: do that, then theoretically we could create a simulation of
Speaker 1: our universe on a smaller scale digitally and then be
Speaker 1: able to run interesting numbers on you know, what would
Speaker 1: have happened if we had tweaked this protein in this protozoa,
Speaker 1: Or what would have happened if there had been more
Speaker 1: antimatter particles rather than matter particles, or what would have
Speaker 1: happened if yeah, if I mean that that age old question,
Speaker 1: if if someone had gone back and killed Hitler, and yeah,
Speaker 1: to the point where you could actually, you know, theoretically
Speaker 1: create life virtual life, which then raises the question, wait,
Speaker 1: if that is possible, If all of that is possible
Speaker 1: for us to be able to make this, to break
Speaker 1: down the universe into a simulation and make it ourselves
Speaker 1: and be able to watch it, really we will. That means, yes,
Speaker 1: we will one day do that because we're people and
Speaker 1: we're curious and we want to do that, which means
Speaker 1: which means Yeah, that that means that that the highest
Speaker 1: likelihood is that we are in fact living in a
Speaker 1: computer simulation right now, because if it's possible, then we
Speaker 1: will do it, and if we will do it, then
Speaker 1: we probably already have done it, and that we the
Speaker 1: people who are living in this reality right now, are
Speaker 1: in fact living through a computer simulation, which could be
Speaker 1: a computer simulation, which could be a computer simulation. Yeah,
Speaker 1: that's what what I started thinking about when we really
Speaker 1: got into the absurdities. Here is this kind of snake
Speaker 1: eating its own tail sort of thing, like, well, imagine
Speaker 1: you could represent the entire universe somehow is data that
Speaker 1: universe representation would have to include all of the simulations
Speaker 1: and data within the universe. Um. Well, if you were
Speaker 1: creating a simulation of the universe, you could be selective
Speaker 1: in what you were including and what you weren't claim.
Speaker 1: But if you were trying to build a an actual like,
Speaker 1: it's essentially like making a map to scale in a
Speaker 1: one to one scale. Yeah, Like, here's my map of
Speaker 1: Atlanta at one to one scale. It's the size of Atlanta.
Speaker 1: Not very useful, Um, but anyway, this is this is
Speaker 1: a philosophical argument that's been made before about whether or
Speaker 1: not we're in a computer simulation, which really was more
Speaker 1: about the idea that we probably will never get there
Speaker 1: in the sense that you know, it wasn't It wasn't
Speaker 1: that we definitely are living in a computer simulation, but
Speaker 1: rather that humankind would very likely end its own existence
Speaker 1: before reaching a point where we were capable of doing
Speaker 1: such a thing. And you're talking about the point where
Speaker 1: you're actually harnessing the power of stars themselves in order
Speaker 1: to generate computer power you need. Well, that's pretty cool,
Speaker 1: and I'm not so much skeptical about that as I
Speaker 1: am about housing consciousness inside a you know, a computer processor, right, Well,
Speaker 1: and and again we this is kind of getting way
Speaker 1: off track, but but but there's the idea that if
Speaker 1: you were able to create a simulation of a human brain,
Speaker 1: there's no way of predicting whether or not it would
Speaker 1: develop its own consciousness. Yeah, we don't know, because we
Speaker 1: haven't been able to build a human brain on a
Speaker 1: real time, uh scale, like a one to one scale
Speaker 1: without you know, we we've built very small models that
Speaker 1: could run in very slow amount of time. But well,
Speaker 1: and and again, I mean, much like the universe we
Speaker 1: really don't know what's going on inside the human brain.
Speaker 1: There's so much of it that we don't understand simulating
Speaker 1: the brain. That's it is big data. Yeah, so anyway, uh,
Speaker 1: you know that, I guess that kind of wraps up
Speaker 1: this whole overview of what big data is and why
Speaker 1: it's And I know I keep saying big data and
Speaker 1: switching to big data, but that's what I do every day.
Speaker 1: But but yeah, this is big business, is what it
Speaker 1: really boils down to, because companies are are trying to
Speaker 1: harness all this information to make it meaningful in some way.
Speaker 1: If if it weren't possible to do that, then we'd
Speaker 1: probably see a lot of these services die off pretty
Speaker 1: quickly because there just wouldn't be the support there to
Speaker 1: financially to have them continue. They make money by serving advertising,
Speaker 1: right right, there's the advertising, and then there's you know,
Speaker 1: there's some companies that are not revenue supported, but they
Speaker 1: are supported by investors, and a lot of these these
Speaker 1: investors are saying, look, I know that right now there's
Speaker 1: no direct way that this service is making money, but
Speaker 1: the data it generates is intrinsically valuable, and as soon
Speaker 1: as we figure out a way of leveraging that data.
Speaker 1: We make all of our investments back, So I mean,
Speaker 1: you know, it's a it's a money game to step
Speaker 1: for profits. It's exactly all right. Well, that wraps up
Speaker 1: this conversation about big data and underpants gnomes. If you
Speaker 1: guys have any suggestions for future episodes of forward Thinking,
Speaker 1: I recommend you get in touch with us. Send us
Speaker 1: an email. That's FW Thinking at discovery dot com, or
Speaker 1: go to f W thinking dot com. Check out our blogs,
Speaker 1: check out our podcasts, check out the social media, check
Speaker 1: out all of the links we have there. We've got
Speaker 1: some really cool content that we want to share with
Speaker 1: you guys, and we want you to be part of
Speaker 1: the conversation. So come on and join us and we
Speaker 1: will talk to you again really soon. For more on
Speaker 1: this topic and the future of technology, visit forward thinking
Speaker 1: dot com, brought to you by Toyota. Let's go Places,