Speaker 1: Welcome to tech Stuff, a production from iHeartRadio. Hey there,
Speaker 1: and welcome to tech Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with iHeart Podcasts. And how the
Speaker 1: tech are you? Now? There's a pretty good chance that
Speaker 1: you've heard or read something about AI chips. But what
Speaker 1: the heck is an AI chip? Is it a microchip
Speaker 1: that actually has artificial intelligence incorporated directly into the semiconductor
Speaker 1: material somehow? And if so, what does that mean? I
Speaker 1: figured it would be a good idea to talk about
Speaker 1: microchips and processors and AI enabled chips in particular to
Speaker 1: help demystify everything because part of the problem, I think
Speaker 1: is that AI chips are are kind of becoming a
Speaker 1: marketing term. It's not just a way to describe technology.
Speaker 1: It's a way to try and set aside a product
Speaker 1: to try and you know, pose it as the new hotness.
Speaker 1: And yes, I know I'm ancient and I use outdated slang.
Speaker 1: So first, when I talk about microchips, I'm talking about
Speaker 1: integrated circuits. Jack Kilby invented the first integrated circuit back
Speaker 1: in nineteen fifty eight at Texas Instruments, and an integrated
Speaker 1: circuit is a collection of interconnected electronic components that happens
Speaker 1: to be built on top of a semiconductor material. Semiconductors,
Speaker 1: as the name suggests, our materials that under certain conditions
Speaker 1: will conduct electricity and under other conditions will insulate or
Speaker 1: block the flow of electricity. The invention of the transistor,
Speaker 1: in addition to the integrated circuit is what allowed for miniaturization.
Speaker 1: That's why computers no longer have to be huge, you know.
Speaker 1: I'm talking about like those old computers, those mainframes that
Speaker 1: would fill up entire rooms or even an entire floor
Speaker 1: of a building. Miniaturization would eventually allow for the production
Speaker 1: of powerful personal computers that were a fraction of the
Speaker 1: size of their predecessors, but just as powerful or even
Speaker 1: more so. The development of arithmetic logic units or ALUs, which,
Speaker 1: as the name suggests, are circuits designed to perform arithmetic
Speaker 1: or mathematical functions on inputs and then produce the relevant outputs. Those,
Speaker 1: in turn served as a building block for the development
Speaker 1: of central processing units or CPUs. The first CPU microprocessor,
Speaker 1: arguably was Intel's for zero zero four computer on a CHEP.
Speaker 1: So this was a fairly limited processor, particularly if we
Speaker 1: judge it by today's standards. I think it had like
Speaker 1: a four bitwidth bus that would allow for processing data,
Speaker 1: which means it could not handle very large values. But
Speaker 1: you have to start somewhere, and the four zero zero
Speaker 1: four was a stepping stone to Intel's eight zero zero
Speaker 1: eight processor. That was the processor that was found in
Speaker 1: a lot of the first commercial personal computers. They used
Speaker 1: the eight zero zero eight processor as their CPU. Now,
Speaker 1: a central processing unit's job is more complex than that
Speaker 1: of an ALU. In fact, ALUs are part of a CPU.
Speaker 1: They are a component that make up part of a CPU.
Speaker 1: The CPU's job is to accept incoming instructions from programs,
Speaker 1: to retrieve those instructions, to execute those instructions on data,
Speaker 1: and to produce the relevant outcomes. And they carry out
Speaker 1: logic operations. They send results to the appropriate destination. That
Speaker 1: destination might be feeding back into software to continue that process,
Speaker 1: or it might mean that you're feeding output to some
Speaker 1: sort of output device like a display or a printer
Speaker 1: or something along those lines. CPUs have two very broad
Speaker 1: categories of operations. Again, this is super super high level.
Speaker 1: I mean, we could get far more complicated than this,
Speaker 1: but they have logic functions and memory functions. Memory being
Speaker 1: that's where you store information so that you can reference
Speaker 1: it quickly in order to carry out these operations, and
Speaker 1: logic being the actual logic gates that end up defining
Speaker 1: how data gets processed. Those are the two big components,
Speaker 1: and there are a few different ways that we measure
Speaker 1: CPU performance. One is we measure it by clock speed.
Speaker 1: You can think of this as the number of instructions
Speaker 1: the CPU is able to handle every second. So the
Speaker 1: higher the clock speed, the more instructions the CPU is
Speaker 1: able to handle per second. That like three point six
Speaker 1: gigahertz would mean three point six billion operations or instructions
Speaker 1: I should say per second. You can also have operations
Speaker 1: that have multiple sets of instructions, so it's a little
Speaker 1: more complicated than just saying, oh, it can handle this
Speaker 1: number of operations per second. Now, you can also have
Speaker 1: CPUs that have multiple cores, and a core is essentially
Speaker 1: all the little individual components of a CPU compartmentalized so
Speaker 1: that you have almost like multiple CPUs on a single chip.
Speaker 1: A single core processor is like a really fast processor.
Speaker 1: A multicore processor is one that divides the processor capabilities
Speaker 1: into these individual cores, and you might wonder, well, why
Speaker 1: would you want to do that, Why would you want
Speaker 1: to take something that is typically very powerful and very
Speaker 1: fast and then divide that up into smaller units. Well,
Speaker 1: that's because some computational processes are able to be performed
Speaker 1: in parallel. This means you can divide up a task
Speaker 1: into smaller jobs and then assign those smaller jobs to
Speaker 1: individual cores. So for these kinds of processes, a multi
Speaker 1: core processor can sometimes be faster than a more powerful
Speaker 1: single core processor would, And that means it's time to
Speaker 1: use an analogy. I bust out every time I talk
Speaker 1: about parallel processing. Fans of tech stuff who have been
Speaker 1: around for years, you all know what's going to happen.
Speaker 1: Go ahead and make yourself a cup of coffee or something.
Speaker 1: So imagine you have a math class. You're a teacher.
Speaker 1: You've got a math class, and your math class has
Speaker 1: five students in it. It's very small focus group. One
Speaker 1: of those five students is like a super math genius.
Speaker 1: They are leagues ahead of the other students. The other
Speaker 1: four students are good at math, they're great students, but
Speaker 1: they take a little more time than the genius does
Speaker 1: to work out Your typical math problem. So you decide
Speaker 1: you're going to give a pop quiz to your class.
Speaker 1: But this pop quiz is a race. It's a race
Speaker 1: that is pitting the super genius against the other four students. Now,
Speaker 1: if that pop quiz consisted of just one mathematical problem,
Speaker 1: or if it had a series of math problems, but
Speaker 1: those math problems were sequential, which means like the information
Speaker 1: you need to solve question two can be found in
Speaker 1: the answer of question one. If that were the case,
Speaker 1: your super genius is gonna win, right because they would
Speaker 1: be able to solve the problem or series of problems
Speaker 1: much more quickly than anyone in the rest of the class.
Speaker 1: And you can't divide that problem up. If it's a
Speaker 1: series that you know question two depends on the outcome
Speaker 1: of question one, you can't divide that up because you
Speaker 1: wouldn't have the information you need to work on the
Speaker 1: problem until the first part was solved. However, let's say
Speaker 1: instead you make a pop quiz that has four math
Speaker 1: problems on it. Each of these four math problems is
Speaker 1: self contained. They do not depend upon the outcomes of
Speaker 1: any of the other questions. So the super genius needs
Speaker 1: to finish all four problems. But for your other four students,
Speaker 1: they're given the option that they can each tackle a
Speaker 1: different problem on the quiz, and if all four of
Speaker 1: them finish whatever respective problem they've chosen first, then as
Speaker 1: a group they win. Now, in that case, the four
Speaker 1: students are far more likely to come out on top.
Speaker 1: The super genius could be as far as like question
Speaker 1: three or four, But each of the other students only
Speaker 1: has to solve a single problem in order to complete
Speaker 1: the pop quiz. That's like a multi core processor working
Speaker 1: on a parallel processing problem. For some subsets of computational operations,
Speaker 1: having multiple cores to work on things all at the
Speaker 1: same time is a big advantage, all right. So that's
Speaker 1: a super high level look at CPUs. Now let's turn
Speaker 1: to GPUs. These are graphics processing units. The name actually
Speaker 1: comes from the g Force two fifty six graphics card
Speaker 1: from Nvidia. So in the nineteen nineties we saw the
Speaker 1: introduction of new graphics intensive applications, particularly in things like
Speaker 1: video editing or in video games, and the CPUs of
Speaker 1: that era were not necessarily optimized to get the job done,
Speaker 1: like it was more work than the CPU could typically handle.
Speaker 1: So the performance of these kinds of programs would be substandards.
Speaker 1: Sometimes the programs wouldn't even run on a computer that
Speaker 1: just had a CPU, even a good CPU. So then
Speaker 1: you had companies like in Video that began to introduce
Speaker 1: graphics cards, and these graphics cards had integrated circuits that
Speaker 1: were better designed. They were optimized to handle graphics processing specifically,
Speaker 1: so that would let the CPU offload the graphics processing
Speaker 1: jobs to the graphics card. The CPU could then focus
Speaker 1: on other operations. The g Force two fifty six introduced
Speaker 1: a ton of new capabilities and features. And while the
Speaker 1: graphics processing unit name might have just started off as
Speaker 1: kind of a marketing strategy, you know, Nvidia gave the
Speaker 1: g Force two fifty six this designation of graphics processing
Speaker 1: unit to kind of set it apart from other graphics
Speaker 1: cards that were on the market. Well, it would turn
Speaker 1: out that the GPU name would have staying power, and
Speaker 1: today any self respecting gamer has a powerful GPU in
Speaker 1: their gaming rig. The GPUs would grow to be more
Speaker 1: important than CPUs, at least for some people. Though it
Speaker 1: would be reductive to say that gamers only need a
Speaker 1: powerful GPU and they don't have to worry about the
Speaker 1: CPU at all. It honestly depends a lot on the
Speaker 1: types of games they play. That is a big component.
Speaker 1: Sometimes having a really fast GPU isn't going to help
Speaker 1: you out that much. It all depends on the types
Speaker 1: of processing you're doing. If you're not doing a lot
Speaker 1: of parallel processing, then a really fast GPU isn't likely
Speaker 1: to boost your performance that much. But the real purpose
Speaker 1: of a GPU is to perform certain types of computational
Speaker 1: operations very quickly and efficiently, in order to do stuff
Speaker 1: like speed up image creation, video and animation. As it
Speaker 1: would turn out, GPUs would also be handy for other things.
Speaker 1: So your typical GPU consists of many specialized processor cores.
Speaker 1: These cores are not designed to do everything you know.
Speaker 1: They do a subset of operations really well, but if
Speaker 1: you ask a GPU to do something outside of that,
Speaker 1: it's not going to perform at you know, at the
Speaker 1: same level as your typical CPU would. But this does
Speaker 1: mean a GPU is a fantastic tool for specific operations
Speaker 1: and then less useful for others. Apart from processing graphics,
Speaker 1: GPUs have been found to be really useful in applications
Speaker 1: ranging from machine learning projects to proof of work cryptocurrency
Speaker 1: mining operations. Now to be clear, GPUs, at least until recently,
Speaker 1: occupied a kind of a sweet space in crypto minds.
Speaker 1: They are not the top of the heap when it
Speaker 1: comes to crypto mining integrated circuits. We'll get to the
Speaker 1: kind that are used in high end crypto mining in
Speaker 1: just a little bit. So for stuff like Bitcoin, which
Speaker 1: as I record this episode, is trading at around fifty
Speaker 1: eight thousand dollars per coin. In fact, I think it's
Speaker 1: like fifty eight point five thousand. That's a lot of money. Well,
Speaker 1: if you're using GPUs, you're not going to cut it.
Speaker 1: You're not going to compete in that space. GPUs just
Speaker 1: can't operate at a level that would make it feasible
Speaker 1: for you to use them for your mining operations. That's
Speaker 1: because the value of bitcoin is so high that it
Speaker 1: drives cryptocurrency miners to seek out the absolute top tier processors,
Speaker 1: and GPUs, while they're great, they're really more mid tier.
Speaker 1: Now it helps if you know what proof of work
Speaker 1: crypto mining is all about. So with proof of work systems,
Speaker 1: you have a network of machines that make up this
Speaker 1: cryptocurrency network, such as bitcoin. We'll use Bitcoin as the
Speaker 1: main example because that was sort of the progenitor of
Speaker 1: this space. So every so often the network issues a challenge,
Speaker 1: which is to solve a mathematical problem, and if you
Speaker 1: do solve it, if you're the first one to solve it,
Speaker 1: you will receive some crypto coins as a reward. The
Speaker 1: act of solving typically is tied to validating a block
Speaker 1: of crypto transactions, So the problem's complexity will depend upon
Speaker 1: a couple of different things. Typically, there's an ideal amount
Speaker 1: of time that it should take to solve this mathematical problem.
Speaker 1: For bitcoin, that time is ten minutes. The other thing
Speaker 1: that determines the complexity of the problem is how much
Speaker 1: computing power is being thrown at solving the problem in
Speaker 1: the first place. So let's go back to our classroom analogy.
Speaker 1: Let's say that you're creating a test, and for whatever reason,
Speaker 1: you have decided this test should take the students ten
Speaker 1: minutes to complete, so you're not really focused on any
Speaker 1: other outcome other than trying to make a test that's
Speaker 1: going to take ten minutes to complete. However, you've misjudged
Speaker 1: the difficulty. Maybe one of your students hands in their
Speaker 1: test six minutes in. Now you know you need to
Speaker 1: make the next test harder in order to hit this
Speaker 1: seemingly arbitrary goal of ten minutes. On the flip side,
Speaker 1: let's say the first student to solve the test took
Speaker 1: fifteen minutes to complete it. Then you know your test
Speaker 1: is too hard and you need to ease up a
Speaker 1: little bit for the next test. When the value of
Speaker 1: cryptocurrency goes up, there's a greater incentive to be the
Speaker 1: first to solve the mathematical problem because the reward is larger.
Speaker 1: That drives miners to buy more processors and to network
Speaker 1: them together, and these are processors that are particularly good
Speaker 1: at solving the types of problems that you get when
Speaker 1: you're crypto mining. For a while, that meant GPUs they
Speaker 1: were the best. But the value of bitcoin went up
Speaker 1: and up and up, and there were other options besides GPUs.
Speaker 1: There were options that were more expensive than GPUs, so
Speaker 1: it require a bigger investment, but then on top of that,
Speaker 1: you were looking at bigger rewards, so it made that
Speaker 1: investment worthwhile. So the integrated circuit that typically replaces GPUs
Speaker 1: for high end cryptocurrency mining, those would be application specific
Speaker 1: integrated circuits or AASAC ASIC. We'll get to those in
Speaker 1: just a bit, so you could if you wanted to
Speaker 1: still run mining rigs using GPUs, nothing would stop you
Speaker 1: from doing that, but you'd be going up against people
Speaker 1: with networks and machines running much more streamlined optimized processors,
Speaker 1: so you would be unlikely to beat them. Okay, we
Speaker 1: got a lot more to cover, but let's take a
Speaker 1: quick break to thank our sponsors. Okay, we're coming back
Speaker 1: to talk a little bit more about crypt currency mining
Speaker 1: in GPUs. So for a while, people who were crypto
Speaker 1: mining ethereum would stick with GPUs. The reason for this
Speaker 1: is ethereum had a lower value, much lower than bitcoin.
Speaker 1: All right, We're talking about a few thousand dollars as
Speaker 1: opposed to tens of thousands of dollars per coin, and
Speaker 1: this meant that it would be impractical to use high
Speaker 1: end integrade circuits like AASC circuits for mining ethereum because
Speaker 1: the cost of doing so would be such that you
Speaker 1: wouldn't be making up that cost in the profit you
Speaker 1: gained from mining the cryptocurrency. So sticking with GPUs made
Speaker 1: more sense, right because from an economic standpoint, that was
Speaker 1: the sweet spot. However, then Ethereum switched to proof of
Speaker 1: steak instead of proof of work. Proof of steak doesn't
Speaker 1: do that whole math problem thing at all, and the
Speaker 1: demand for GPUs and crypto mining plummeted as a result.
Speaker 1: There are other cryptocurrencies out there, some of which that
Speaker 1: still do use proof of work, but they're not as
Speaker 1: sought after as Bitcoin or ethereum are. So this meant
Speaker 1: that the demand for GPUs in the crypto space began
Speaker 1: to diminish, and that became really good news for people
Speaker 1: who wanted a GPU for something else, like for a
Speaker 1: gaming rig for example. Now, I would say for the
Speaker 1: majority of people out there, like your average consumers, CPUs
Speaker 1: and GPUs are the beginning and end of it when
Speaker 1: it comes to processors or microchips that are meant to
Speaker 1: act like processors, But there are a couple of other
Speaker 1: varieties out there that we use for special purposes. And
Speaker 1: the special purpose thing is the important part to keep
Speaker 1: in mind. A CPU, by necessity, has to be able
Speaker 1: to do a bit of everything right. Because a CPU
Speaker 1: is the control center of your typical computer. It needs
Speaker 1: to be able to handle operations from a variety of
Speaker 1: different programs and that kind of thing. It is a
Speaker 1: jack of all trades master of none. It needs to
Speaker 1: be able to handle whatever you throw at it, but
Speaker 1: that means it cannot be optimized for any specific task.
Speaker 1: So what it lacks in efficiency, it makes up for inversatility.
Speaker 1: GPUs are more specialized and so they can handle certain
Speaker 1: processes better than a CPU typically can, But a GPU
Speaker 1: might not be so good at executing all the different
Speaker 1: tasks that a CPU has to handle, So while it
Speaker 1: is faster with some stuff, it's slower with other stuff. Now,
Speaker 1: the next two types of semiconductor devices I want to
Speaker 1: mention are even more specialized than GPUs, and then we'll
Speaker 1: end with one that is specialized specifically for the AI field.
Speaker 1: So next up is the field programmable gait arrays or
Speaker 1: FPGA's now a definition from XLinks dot com because x
Speaker 1: links is what introduced this technology back in the mid
Speaker 1: nineteen eighties. So x links defines this as FPGA's quote.
Speaker 1: Are based around a matrix of configurable logic blocks CLBs
Speaker 1: connected via programmable interconnects. End quote that sounds like gibberish
Speaker 1: to some folks. It's definitely got some barriers there from
Speaker 1: easy understanding, but the idea is pretty simple. When you
Speaker 1: boil it down to what's basically happening. So imagine that
Speaker 1: you have a microchip and you're able to reconfigure the
Speaker 1: individual components on that microchip so that they're optimized for
Speaker 1: whatever it is you need to do. So you can
Speaker 1: reprogram this chip, in other words, so that it is
Speaker 1: better aligned with the task you have at hand. As
Speaker 1: I said, x links first introduced this type of integrated
Speaker 1: circuit back in nineteen eighty five, and the aim us
Speaker 1: to make an integrated circuit that could potentially fit the
Speaker 1: needs of different specific use cases, not by being a
Speaker 1: jack of all trades that could do anything, but do
Speaker 1: so at a kind of a mediocre level, but rather
Speaker 1: by being configured to work best for that specific application. Moreover,
Speaker 1: you can at least sometimes reconfigure without having to change
Speaker 1: the actual physical architecture of the chip itself. This is
Speaker 1: important because not everyone has access to a clean room
Speaker 1: with incredibly precise and computer operated tools. That's exactly what
Speaker 1: you would need if you wanted to perform surgery on
Speaker 1: a microchip. Instead, an FPGA has these CLBs that x
Speaker 1: links talked about, the configurable logic blocks. These can be
Speaker 1: programmed to act like simple logic gates, and these gates
Speaker 1: follow specific rules. Essentially, they either allow electrical current to
Speaker 1: flow through or they block it from flowing through, and this,
Speaker 1: when you look at it a macro level, is what
Speaker 1: allows operations on a processor. The field and field programmable
Speaker 1: gate array means you can actually do this reprogramming after
Speaker 1: the FPGA has shipped from its manufacturer. So instead of
Speaker 1: working with a manufacturer to specialize a chip from the
Speaker 1: design phase and then go all the way through to manufacturing,
Speaker 1: the manufacturer makes this FPGA that can potentially be one
Speaker 1: of thousands of different configurations, and then you program it
Speaker 1: once you receive it. Now, some of these FPGAs are
Speaker 1: limited to kind of a one time only configuration, so
Speaker 1: you can program them once you get them, but then
Speaker 1: they're set in that particular configuration from that point forward.
Speaker 1: But others are designed so that they can be reprogrammed
Speaker 1: multiple times, which obviously makes them very useful. If you
Speaker 1: wanted to prototype a technology and you aren't really sure
Speaker 1: which configuration is going to be best for whatever it
Speaker 1: is you're trying to do, it's great to have a
Speaker 1: chip you can reprogram so you can try different configurations
Speaker 1: to find the one that makes the most sense for
Speaker 1: whatever it is you're trying to achieve. One issue with
Speaker 1: FPGA's is that they are not cost efficient when you're
Speaker 1: looking at mass production. They're great if you are prototyping,
Speaker 1: but if you plan to make a whole bunch of them,
Speaker 1: it gets time consuming and expensive because not only do
Speaker 1: you have to have them made, then you have to
Speaker 1: have them programmed. Plus sometimes you may have an application
Speaker 1: in mind that an FPGA cannot accommodate even with all
Speaker 1: the reconfiguring. So think of an FPGA as having a
Speaker 1: limited number of configurations and it turns out that what
Speaker 1: you need is outside of this range. That would mean
Speaker 1: you would need to add additional integrated circuits to your
Speaker 1: system to accommodate these limitations of the FPGA itself, which
Speaker 1: means you're adding more complexity to your system, and that
Speaker 1: in turn also means you're adding more costs to your system.
Speaker 1: Next up, we have the one I mentioned earlier, the
Speaker 1: Application specific integrated circuit or AZC ASIC, as the name indicates,
Speaker 1: These chips are made to operate for specific applications, and
Speaker 1: as such, they are highly optimized from the hardware level
Speaker 1: up for that purpose. They are not meant to be
Speaker 1: general purpose processors like a CPU. So if you put
Speaker 1: an ASK to work on a task that it was
Speaker 1: not designed to handle, you are not going to get
Speaker 1: a good result. In fact, it may not work at all.
Speaker 1: But when it's integrated into a system that's meant to
Speaker 1: do that one thing it was designed to do, it
Speaker 1: does it really well. And AAK can be a speed
Speaker 1: demon and operate at an efficiency that's much more desirable
Speaker 1: than your typical CPU or even GPU. So unlike an
Speaker 1: FPGA and ASK cannot be reconfigured. It is a high tech,
Speaker 1: highly specialized chip. There are a few different approaches to
Speaker 1: create that specialization during the manufacturing process, but I feel
Speaker 1: like that's beyond the scope of this episode. I'll save
Speaker 1: it for a time when I do a full episode
Speaker 1: about AZK chips. Now, the design process for AZIK is complicated.
Speaker 1: So imagine you're building a chip intended to do one
Speaker 1: thing extremely well. You would have to do a lot
Speaker 1: of work to make sure that the chip you were
Speaker 1: designing met that purpose. So that means there's a lot
Speaker 1: of R and D and there's a lot of testing. However,
Speaker 1: once you do arrive at this final design, one big
Speaker 1: advantage of AZK over FPGA is that it can then
Speaker 1: go into large volume production. So while the development process
Speaker 1: of an AZK is typically longer and more expensive than
Speaker 1: using an FPGA, once you do get to the production stage,
Speaker 1: the AZIC chips become more cost effective. So if you're
Speaker 1: doing a one off, FPGA makes the most sense financially.
Speaker 1: If your goal is to make something that you're going
Speaker 1: to mass produce, AZIC makes far more sense. ASIC chips
Speaker 1: also tend to be more power efficient than FPGA's, so
Speaker 1: by their nature, an FPGA needs to have components that
Speaker 1: aren't necessary for all applications because the whole point of
Speaker 1: an FPGA is that you can reprogram them to do
Speaker 1: specific tasks, but not every task is going to need
Speaker 1: every component that's on that circuit. So that means there's
Speaker 1: going to be some extra stuff on that integrated circuit
Speaker 1: that ends up being superfluous for certain operations. With AZC,
Speaker 1: you can leave off anything that would be superfluous, right,
Speaker 1: You can leave that out of the design because you
Speaker 1: know ahead of time what you're putting this chip to
Speaker 1: work for, so you can only focus on the things
Speaker 1: that are absolutely necessary for the operation of that chip.
Speaker 1: That means you don't have to supply power to components
Speaker 1: that aren't actually doing anything. That keeps your power consumption
Speaker 1: costs lower in the long run. Thus, ASIC chips are
Speaker 1: more efficient. Now, most of us are not going to
Speaker 1: be shopping around for ASK chips. Your average consumer has
Speaker 1: no need for them. But for folks like cryptomners, AZK
Speaker 1: might make sense once you reach a certain level of profit. Right,
Speaker 1: once you reach a certain level at least potential profit
Speaker 1: if you mine a block of the cryptocurrency. Because again,
Speaker 1: Bitcoin created the perfect storm for this back when it
Speaker 1: was awarding six point two five coins per block, so
Speaker 1: that meant in an average day the system would release,
Speaker 1: or rather miners would mine around nine hundred bitcoins total
Speaker 1: per day, and with bitcoins trading at fifty grand each,
Speaker 1: that would mean around forty five million dollars worth of
Speaker 1: bitcoin were up for grabs every single day. That's what
Speaker 1: justified spending the huge amount of money it costs to
Speaker 1: develop and deploy ACC chips for the specific task of
Speaker 1: mining bitcoin. Yes, that design process is incredibly expensive, but
Speaker 1: if you could create a system that could grab a
Speaker 1: significant number of bitcoins every day, then it would pay
Speaker 1: for itself pretty darn quickly. You might not get all
Speaker 1: the bitcoins, you might not even get most of them,
Speaker 1: but as long as you were grabbing a decent number
Speaker 1: every single day, you would quickly accumulate wealth and justify
Speaker 1: the cost of using ACAC technology. That's what left GPU
Speaker 1: miners in the dust, because once acc systems joined the party,
Speaker 1: the GPUs just could not compete. It would be kind
Speaker 1: of like if you put me in the one hundred
Speaker 1: meter dash in the Olympics, the lead runner would be
Speaker 1: crossing the finish line before I managed to get a
Speaker 1: quarter of the way there. Now I should add that
Speaker 1: this year, in twenty twenty four, the number of bitcoins
Speaker 1: awarded per block dropped by half. This was all part
Speaker 1: of the plan. This wasn't a mistake or something. Now,
Speaker 1: if you mine a block, instead of getting six point
Speaker 1: twenty five coins, you end up getting three point one
Speaker 1: two five. So again, this was this was planned, and
Speaker 1: every four years or so the system cuts the number
Speaker 1: of coins awarded per block mind by fifty percent. When
Speaker 1: bitcoin first hit the scene back in early two thousand
Speaker 1: and nine, if you mined a block successfully you would
Speaker 1: net yourself fifty bitcoins per pop. But of course, back
Speaker 1: in two thousand and nine, the value of bitcoin was
Speaker 1: fractions of a cent. You wouldn't apply AASAC technology to
Speaker 1: bitcoin mining back in those days because the coins weren't
Speaker 1: really worth anything. In fact, on May twenty second, twenty ten,
Speaker 1: this is a famous date in crypto history. This was
Speaker 1: more than a year after bitcoin had launched. A cryptocurrency
Speaker 1: minor named Laslow's spent ten thousand bitcoins in order it
Speaker 1: to order ap pizza. So today that pizza would be
Speaker 1: worth more than five hundred and eighty five million dollars.
Speaker 1: And in fact, another interesting point, Bitcoin is a lot
Speaker 1: of volatility. When I started work on this episode, it
Speaker 1: was trading at fifty seven thousand dollars and now it's
Speaker 1: at more than fifty eight thousand, so the value changes
Speaker 1: pretty drastically. Anyway, getting back to the having, part of
Speaker 1: the bitcoin strategy is that there's a finite number of
Speaker 1: bitcoin that will ever be released into circulation, and once
Speaker 1: the last one is in circulation, no more new bitcoin
Speaker 1: will be minted. So specifically, that makes up twenty one
Speaker 1: million bitcoin to control the release of bitcoin into circulation.
Speaker 1: The system does this having business every four years, so
Speaker 1: today mining a block on the Bitcoin network will earn
Speaker 1: you three point one two five bitcoins, or around one
Speaker 1: hundred and eighty one thousand dollars worth of bitcoin crypto
Speaker 1: per block. Mind, these kinds of changes affect mining operations
Speaker 1: because if the magic number dips too much, then it
Speaker 1: would cost more to mine bitcoin. Then you would get
Speaker 1: out of mining it, so you would have to adjust
Speaker 1: your strategy. Right, you'd say, all right, well, now it
Speaker 1: doesn't make sense for me to operate this massive computer
Speaker 1: network of AASC machines that's drawing power directly from a
Speaker 1: formerly decommissioned power plant because the cost of operations is
Speaker 1: sky high and the amount that I'm able to actually
Speaker 1: mine is much lower. Now I have one more initialism
Speaker 1: to throw your way, but before we get to that,
Speaker 1: let's take another quick break to thank our sponsors. Okay,
Speaker 1: we're back, and we've talked about CPUs, and we've talked
Speaker 1: about GPUs, and we've talked about FPGAs talked about ASICs.
Speaker 1: Now it's time to talk about NPUs. And as a nancy,
Speaker 1: the initialism stands for neural processing unit. These have technically
Speaker 1: been around for a few years now, but the term
Speaker 1: is still fairly new. For mainstream audiences. I think you
Speaker 1: started to see them pop up in mainstream tech journals
Speaker 1: last year, but they've been around for a few years.
Speaker 1: And NPU is a chip with a specialized design meant
Speaker 1: for AI applications and artificial neural networks in particular. Now,
Speaker 1: just in case artificial neural networks, if that term is
Speaker 1: new to you, it is a network of processors that
Speaker 1: collectively mimics the way our neurons interconnect with one another
Speaker 1: in our brain meet. That's a very high level and
Speaker 1: oversimplified explanation, but it kind of gets the idea across.
Speaker 1: Artificial neural networks are often used in the field of
Speaker 1: machine learning, in which researchers train computer system to produce
Speaker 1: specific results given specific input. Now, that could be as
Speaker 1: simple as indicating which of a million different photographs are
Speaker 1: the ones that happen to have cats in them versus
Speaker 1: ones that don't have cats in them, or it could
Speaker 1: be something far more complicated, like learning which environmental factors
Speaker 1: impact the development of weather systems so that you can
Speaker 1: have a more accurate weather forecast. And NPU is tuned
Speaker 1: to work in this discipline, and often it could produce
Speaker 1: much better results than a GPU. Both an NPU and
Speaker 1: a GPU tend to be made with parallel processing in mind,
Speaker 1: and NPUs are typically incorporated onto integrated circuits that also
Speaker 1: have a CPU. They don't necessarily replace a CPU, they
Speaker 1: are in addition to one. So let's wrestle this all
Speaker 1: back to artificial intelligence. When you hear the phrase AI chip,
Speaker 1: chances are the chip question is one of four types.
Speaker 1: It's an FPGA, an ASIC, an NPU, or a GPU.
Speaker 1: Now you can have AI enabled CPUs that don't have
Speaker 1: these other components. But the problem with CPUs is that
Speaker 1: due to their unspecialized design, they have limited usefulness when
Speaker 1: it comes to AI applications, particularly as the AI field
Speaker 1: becomes more sophisticated and has greater data processing needs. It's
Speaker 1: kind of like giving a really good third year math
Speaker 1: student a challenging quiz meant for fifth year students. Our
Speaker 1: little test subject might do a decent job at the
Speaker 1: end of the day, but it will likely take them
Speaker 1: longer and cause more exertion than it would for someone
Speaker 1: who is more attuned to the task. So with CPUs,
Speaker 1: that means that you have to have longer processing times
Speaker 1: and you have to use more energy in order to
Speaker 1: be able to complete the task, and that means also
Speaker 1: generating more heat. It's less efficient, it's less money efficient
Speaker 1: as well, not just power efficient, but financially efficient. So
Speaker 1: two of the components you find on these integrated circuits
Speaker 1: are logic gates and we could just call them transistors
Speaker 1: for simplicity, and then memory. So while a CPU depends
Speaker 1: on both of these quite a bit in order to
Speaker 1: do its job, specialized chips like ASICs AASEYS can be
Speaker 1: made to emphasize the logic components more than the memory components,
Speaker 1: and they can be packed with more transistors with less
Speaker 1: space reserved for memory. That's typically what AI needs needs
Speaker 1: access to large capacity for data processing, so the goal
Speaker 1: is to allow for more data processing per unit of
Speaker 1: energy than you would get out of a typical microchip.
Speaker 1: AI is a power hungry technology. I mean that literally.
Speaker 1: Maybe one day AI will be power hungry in the
Speaker 1: figurative sense, like in like the super villain sense. Maybe
Speaker 1: that will happen one day, but right now, it's just
Speaker 1: it needs a lot of juice. So making the processing
Speaker 1: as efficient as possible is absolutely vital, Otherwise the costs
Speaker 1: of operations spiral out of control. Your energy needs as
Speaker 1: well as things like cooling needs and everything else that
Speaker 1: goes along with using a bucket load of power would
Speaker 1: make it harder for you to cover costs. This is
Speaker 1: part of the reason why you'll hear about companies spending
Speaker 1: billions of dollars on AI. It's not just that they
Speaker 1: have to spend that money for the research and development
Speaker 1: of AI, although that takes up a big part of it.
Speaker 1: It's that actually operating these data centers that are running
Speaker 1: these specialized machines takes a lot of energy, and so
Speaker 1: the cost of operation is in the billions of dollars. Now,
Speaker 1: these AI chips typically can handle parallel processing tasks in
Speaker 1: a much greater capacity than even your most powerful multi
Speaker 1: thread into multi core CPUs can. Which type of chip
Speaker 1: you use often depends upon the application you want, so,
Speaker 1: for example, Google's tensor processing Unit is an ASK chip.
Speaker 1: Google has spent a lot of time and money developing
Speaker 1: these processors and fine tuning them to handle intense data
Speaker 1: processing at incredible speed for the purposes of machine learning applications. Primarily,
Speaker 1: a lot of AI companies will use off the shelf
Speaker 1: GPUs and they will wire them together in order to
Speaker 1: train AI models, which has led to Nvidia, which for
Speaker 1: years was thought of as just a graphics processing unit
Speaker 1: design company, to now become a leading AI chip company.
Speaker 1: The boom in AI development has catapulted Nvidia to become
Speaker 1: a three trillion dollar company in recent years, so it
Speaker 1: has joined the likes of Microsoft and Apple. That's not
Speaker 1: to say Nvidia was always like an underdog or anything.
Speaker 1: It was always a company that was doing pretty darn well,
Speaker 1: but in recent years it has entered the stratospheric level evaluation.
Speaker 1: It was not a trillion dollar company that long ago.
Speaker 1: When we talk about consumer products, CPUs and NPUs are
Speaker 1: typically what will handle AI needs because they are the
Speaker 1: more cost efficient approach. Intel has developed NPUs under the
Speaker 1: code name metior Lake. Actually, to be more precise, the
Speaker 1: metior Lake chips include CPU cores. They also include a
Speaker 1: small GPU portion as well as the NPU unit, all
Speaker 1: on this same integrated circuit. And the idea is that
Speaker 1: these chips will be incorporated into machines that can run
Speaker 1: AI workloads locally. So let's say you've got a company
Speaker 1: and that company wants to host a language model, but
Speaker 1: it wants it locally. It doesn't want to be tapping
Speaker 1: into a cloud based language model, they want to run
Speaker 1: it on premises, while they might use computers with meteor
Speaker 1: Lake chips in them in order to do that processing,
Speaker 1: which would be more cost effective than building out a
Speaker 1: whole AI data center just to service this specific company. Okay,
Speaker 1: so when people talk about AI chips, they don't mean
Speaker 1: that somehow the chips are imbued with artificial intelligence. Instead,
Speaker 1: these chips are optimized to run AI applications, and those
Speaker 1: applications run the entire gamut of AI. There are AI
Speaker 1: chips used in robotics, There are AI chips used in
Speaker 1: autonomous cars. There are AI chips for large language models.
Speaker 1: Smaller chips and NPUs can be incorporated into smart devices,
Speaker 1: which allow some AI processing to happen at the device
Speaker 1: level rather than remotely through a network connection. That's really
Speaker 1: important for speeding up those processes and to eliminate latency,
Speaker 1: because for some implementations speed might not be that big
Speaker 1: a deal, but for others, like the autonomous cars I mentioned,
Speaker 1: being able to process information and produce results is critical
Speaker 1: to operate the technology safely. You cannot have latency in
Speaker 1: those systems or disaster can occur. You wouldn't want an
Speaker 1: autonomous car that constantly has to beam information up to
Speaker 1: the cloud and wait for a response, because real world
Speaker 1: driving conditions are constantly changing. They are dynamic, and they
Speaker 1: change at a very fast rate. Depending on how quickly
Speaker 1: you're driving, it could be an incredibly fast rate. So
Speaker 1: any latency would lead to catastrophic outcomes. So AI chips
Speaker 1: are important components in what you might call EDGEAI. This
Speaker 1: not only cuts down on processing time, but it can
Speaker 1: also help things remain more secure. Right, you're not beaming
Speaker 1: data to a different location all the time, you're processing
Speaker 1: it locally. That makes it less susceptible to being hacked.
Speaker 1: Not immune, but it's less susceptible. There's fewer links in
Speaker 1: the chain, you could say. So now we have our
Speaker 1: overview of what AI chips are all about. And I
Speaker 1: think it's good to remember that a processor's utility depends
Speaker 1: entirely upon what you plan to actually use it for.
Speaker 1: If you're doing standard computing stuff like you're working with
Speaker 1: documents or playing games or browsing the web, and AI
Speaker 1: chip isn't really going to mean much to you at all.
Speaker 1: AI chips tend to be really geared toward parallel processing.
Speaker 1: So it's possible that a computer with a good AI
Speaker 1: chip could be useful as a gaming rig. But honestly,
Speaker 1: I think, at least for now, going with a good
Speaker 1: GPU and a decent CPU matters more for gamers, And
Speaker 1: like I said, some cases, you might not need a
Speaker 1: really good GPU. You could have a decent GPU and
Speaker 1: a really good CPU. It all kind of depends on
Speaker 1: the types of games you want to play. I think
Speaker 1: it's important for regular old folks like me and at
Speaker 1: least some of y'all out there, to know about this
Speaker 1: stuff so that when we're shopping around for our next device,
Speaker 1: we have an understanding of the terminology. Right, we know
Speaker 1: what an AI chip is and what it's supposed to do,
Speaker 1: and whether or not it matches what we need. We
Speaker 1: aren't just pooled by marketing terms. You know. It doesn't
Speaker 1: mean that an AI chip labels slapped on something is
Speaker 1: going to mean that that's the best thing for us.
Speaker 1: So having this understanding is important. Being an informed consumer
Speaker 1: is important. It means you're going to get the best
Speaker 1: out of your money that meets your needs. Right, we
Speaker 1: only have so much money. We should make sure that
Speaker 1: when we're spending it. We're doing it on stuff that
Speaker 1: actually solves the problems we have, as opposed to just
Speaker 1: stuff that's shiny and new. I say this because a
Speaker 1: lot of tech enthusiasts tend to fall into the trap
Speaker 1: of I want the new thing because the new thing
Speaker 1: is somehow better than the old thing. That's not always
Speaker 1: the case. It often is in tech, but it's not
Speaker 1: always the case, and it certainly doesn't always justify spending
Speaker 1: the amount of money it takes to be part of
Speaker 1: that bleeding edge. It is important that we have a
Speaker 1: bleeding edge, but it's not important that we're all in it.
Speaker 1: We can hang back a bit if we need to.
Speaker 1: So I just wanted to take this chance to kind
Speaker 1: of break down this AI chip terminology and what it
Speaker 1: actually means, because goodness knows, Like when I started first
Speaker 1: seeing the terminology myself, I was confused. I was thinking,
Speaker 1: what makes an AI chip and aichip? And does it
Speaker 1: have some sort of AI capability built into it? Because
Speaker 1: how would that work? And obviously I was overthinking it.
Speaker 1: So hopefully this was useful for y'all, and I hope
Speaker 1: you're all doing well, and I'll talk to you again
Speaker 1: really soon. Tech Stuff is an iHeartRadio production. For more
Speaker 1: podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts, or
Speaker 1: wherever you listen to your favorite shows.