WEBVTT - The AI that lives Inside your PC: Revolutionizing Computing 0:00:12.960 --> 0:00:15.480 For as long as humans have used computers, we have 0:00:15.560 --> 0:00:19.480 sought to make them faster, smarter, and more capable. But 0:00:19.520 --> 0:00:22.440 what happens when we harness the power of artificial intelligence 0:00:22.560 --> 0:00:25.639 with new PC hardware and software? And what if every 0:00:25.640 --> 0:00:29.520 computer could use AI technology to unleash new capabilities that 0:00:29.600 --> 0:00:33.080 will benefit anyone and everyone who uses a computer. 0:00:34.479 --> 0:00:35.360 What can I help you with? 0:00:36.440 --> 0:00:39.320 This is not a distant dream. Each day it becomes 0:00:39.400 --> 0:00:43.080 more and more of a reality. AI is no longer 0:00:43.120 --> 0:00:47.360 exclusively running in the cloud. It's increasingly finding its way 0:00:47.400 --> 0:00:51.680 into all computers, enabling business users to do more and 0:00:51.760 --> 0:00:56.320 be more with PC technology from Intel. In this episode, 0:00:56.640 --> 0:01:00.720 we'll be focusing on defining the AIPC and what represents 0:01:00.960 --> 0:01:05.000 for the future of this transformative technology. We'll also explore 0:01:05.040 --> 0:01:07.280 what the growth of AI running directly on the PC 0:01:07.720 --> 0:01:12.480 versus the cloud means and how this AIPC revolution changes 0:01:12.520 --> 0:01:16.640 how we work, live and create. Join us as we 0:01:16.680 --> 0:01:19.440 take a journey into the future of AI computing, where 0:01:19.480 --> 0:01:23.399 machines are not just tools but partners in our endeavors. 0:01:27.400 --> 0:01:32.160 Welcome to Technically Speaking, an Intel podcast produced by iHeartMedia's 0:01:32.200 --> 0:01:37.240 Ruby Studio in partnership with Intel. Hey the I'm gram class. 0:01:37.840 --> 0:01:40.720 Joining us today is Robert Hallock, the VP and General 0:01:40.720 --> 0:01:44.720 manager of Client AI and Technical Marketing at Intel. Here 0:01:44.720 --> 0:01:51.320 has a long history with product development and PCs. Welcome 0:01:51.360 --> 0:01:52.400 to the show, Robert. 0:01:52.360 --> 0:01:53.120 Thanks for having me. 0:01:53.200 --> 0:01:55.280 Good to be here. Yeah, I just want to the 0:01:55.320 --> 0:01:58.200 start off by saying, there seems to be a lot 0:01:58.240 --> 0:02:01.320 of talk around AI, and it seems to always revolve 0:02:01.360 --> 0:02:05.720 around cloud based or software as a service type business models. 0:02:06.360 --> 0:02:09.919 How Intel is taking a different route. Well, but can 0:02:09.919 --> 0:02:13.520 you give us an overview of what an AIPC means 0:02:13.520 --> 0:02:17.120 at Intel and can you help us understand some of 0:02:17.160 --> 0:02:21.080 the benefits of running AI directly on a PC versus 0:02:21.120 --> 0:02:21.760 in the cloud. 0:02:22.480 --> 0:02:25.000 Yeah, a lot of people have been exposed to AI 0:02:25.440 --> 0:02:30.000 through the cloud, and these are services like chat, GPT 0:02:30.440 --> 0:02:34.440 or Dolly three where you can create a piece of 0:02:34.480 --> 0:02:38.280 content from a text description. And that's one kind of 0:02:38.320 --> 0:02:42.240 AI that's called generative AI is a category pros and 0:02:42.280 --> 0:02:44.560 cons to that doing it in the cloud. The pro 0:02:44.880 --> 0:02:48.760 is the models are huge, right. They essentially have the 0:02:48.760 --> 0:02:53.840 Internet's collective knowledge of data to produce an answer or 0:02:53.840 --> 0:02:57.960 a picture, and that's highly detailed. But one of the 0:02:58.000 --> 0:03:01.720 cons is that they're not very specific to you what 0:03:01.840 --> 0:03:04.720 you are doing on your PC, the information on your computer, 0:03:05.040 --> 0:03:08.200 or the images that you care about. So that's sort 0:03:08.240 --> 0:03:12.960 of motivating to bring AI capabilities locally to the PC 0:03:13.200 --> 0:03:18.040 where you don't need an Internet connection to use these capabilities. 0:03:18.240 --> 0:03:20.880 And that's sort of the tip of the iceberg, because 0:03:21.080 --> 0:03:25.840 when you move AI to the PC without a cloud connection, 0:03:25.919 --> 0:03:29.560 you can also do new types of AI workloads that 0:03:29.600 --> 0:03:33.239 would be too big or simply can't run in the cloud. 0:03:33.360 --> 0:03:37.520 So a good example of that is in teleconferencing, lots 0:03:37.560 --> 0:03:41.840 of people use background blurring or background pictures, and we 0:03:41.880 --> 0:03:45.040 can actually make that a lot more energy efficient. We 0:03:45.080 --> 0:03:48.000 can actually give you hours of battery life back by 0:03:48.080 --> 0:03:50.320 using AI on that use case. So the whole point 0:03:50.480 --> 0:03:53.920 of AI is that there's a wave of software coming 0:03:54.280 --> 0:03:58.920 that uses AI to improve performance or to reduce power consumption. 0:03:59.120 --> 0:04:02.080 So you'll get better performance from your system and longer 0:04:02.120 --> 0:04:04.360 battery life as well. And this will continue to grow 0:04:04.440 --> 0:04:05.880 over the next five years. 0:04:06.440 --> 0:04:09.360 And Robert I was wondering if you could please describe 0:04:09.360 --> 0:04:11.560 to the audience what the current state of the art 0:04:11.600 --> 0:04:15.640 is with PC architecture. You know, we've got CPUs, which 0:04:15.640 --> 0:04:19.279 are central processing units that make computing possible, and you've 0:04:19.279 --> 0:04:23.560 got GPUs, which are graphical processing units that are instrumental 0:04:23.560 --> 0:04:28.840 for machine learning, gaming applications, video editing. So how does 0:04:28.839 --> 0:04:31.920 that actually differ from the concept of an AIPC. 0:04:32.640 --> 0:04:35.039 That's a great question. So, yeah, users may have heard 0:04:35.080 --> 0:04:39.239 about an AIPC at this point, and at the basic level, 0:04:39.440 --> 0:04:44.440 this is just a new generation computer with hardware inside 0:04:44.480 --> 0:04:49.039 that is capable of accelerating an AI based workload and 0:04:49.240 --> 0:04:53.520 previous hardware, let's say early twenty twenty three computers, you 0:04:53.560 --> 0:04:57.880 can run AI workloads, but they will fall back to 0:04:57.960 --> 0:05:02.359 the CPU cores, which are not as fast and not 0:05:02.520 --> 0:05:06.520 as energy efficient as running that same workload on the 0:05:06.560 --> 0:05:10.960 new accelerators in an AIPC. And the way we're doing 0:05:10.960 --> 0:05:14.679 it at Intel is actually the CPU cores themselves. We've 0:05:14.720 --> 0:05:19.760 added AI accelerating capabilities to those cores. The GPU that's 0:05:19.760 --> 0:05:23.159 built into our processor also has accelerators, and then we've 0:05:23.160 --> 0:05:28.040 also added an entirely new component called the NPU or 0:05:28.040 --> 0:05:31.640 neural processing unit, which sits next to the CPU and 0:05:31.640 --> 0:05:34.760 the GPU and is now its own third category of 0:05:34.839 --> 0:05:40.479 acceleration on the device. And different workloads in AI, or 0:05:40.560 --> 0:05:44.400 different features in AI run best on one of those 0:05:44.400 --> 0:05:46.719 three engines, so you kind of need all three to 0:05:46.800 --> 0:05:50.400 do this well. And our new product, Intel Core Ultra 0:05:50.760 --> 0:05:54.200 is our first AIPC processor, and so if you see 0:05:54.200 --> 0:05:56.880 that name Core Ultra, you know that it has the 0:05:57.279 --> 0:06:00.360 AI accelerators to run these features well. But at its route, 0:06:00.800 --> 0:06:04.080 it is a PC that has AI specific hardware. 0:06:04.800 --> 0:06:09.200 And I'm quite interested in I guess the thinking behind 0:06:09.400 --> 0:06:13.359 at Intel of this shift towards this new architecture. Was 0:06:13.400 --> 0:06:17.360 it something that was internally driven or did you see 0:06:17.400 --> 0:06:21.080 some conversations and discussions with your key customers and clients 0:06:21.800 --> 0:06:23.719 driving that sort of shift. 0:06:24.000 --> 0:06:28.280 This is very much a software industry driven transition, and 0:06:28.760 --> 0:06:32.120 CPU vendors like Intel were innovating to keep pace with 0:06:32.160 --> 0:06:34.520 what's going on in the software environment. And that's kind 0:06:34.520 --> 0:06:36.240 of a key thing I'd want to stress. You know, 0:06:36.279 --> 0:06:40.040 maybe you're on the fence about AIPC or you wonder, 0:06:40.120 --> 0:06:42.200 you know, how long is this thing going to stick around. 0:06:42.800 --> 0:06:44.440 This is one of those cases where it's both the 0:06:44.480 --> 0:06:48.359 hardware industry and the software industry agreeing that this is 0:06:48.400 --> 0:06:51.960 the right thing to do for performance and features and power. 0:06:52.600 --> 0:06:55.120 Because I truly believe that over the next three to 0:06:55.200 --> 0:06:59.760 five years we will reach this point of general acceptance 0:07:00.120 --> 0:07:02.760 in AI, where it's whether you know it or not, 0:07:03.040 --> 0:07:06.680 widely diffused. In most of the applications you're working on 0:07:06.920 --> 0:07:09.320 or working with, many of the features that you value 0:07:09.440 --> 0:07:13.520 will use AI again transparently in the background. But this 0:07:13.600 --> 0:07:17.920 is very much a collaborative, industry wide effort from all 0:07:17.960 --> 0:07:21.480 the hardware makers, all the big software vendors. This is 0:07:21.520 --> 0:07:25.720 here to stay for sure. Let's pause for second here 0:07:26.000 --> 0:07:29.680 to reiterate Robert's point. Right of adoption is always a 0:07:29.680 --> 0:07:32.880 crucial aspect of any new technology, and that's certainly true 0:07:32.880 --> 0:07:36.400 of what we're discussing today with AIPCS. It's important to 0:07:36.480 --> 0:07:38.960 understand that the leaders in the field of computing, both 0:07:39.000 --> 0:07:42.680 in the hardware and software domain, have already begun down 0:07:42.720 --> 0:07:45.760 this revolutionary path and it doesn't seem like there's any 0:07:45.880 --> 0:07:48.920 roadblocks in site. With that in mind, I asked Robert 0:07:48.960 --> 0:07:51.840 about the benefits of moving AI workloads from the cloud 0:07:52.160 --> 0:07:54.840 to the PC, especially when it comes to issues of 0:07:54.920 --> 0:08:00.600 data privacy, latency, and connectivity. You just touched couple that 0:08:00.640 --> 0:08:03.320 are really important to local AI. It's that data privacy 0:08:03.400 --> 0:08:07.400 or data security. We've all read about information going up 0:08:07.440 --> 0:08:10.520 to a cloud resource of some kind, and you don't 0:08:10.560 --> 0:08:13.880 really know what's going to happen with your data or 0:08:13.880 --> 0:08:16.400 your request after that. And that's not to say there's 0:08:16.440 --> 0:08:20.440 anything malicious implied. You just don't know. So I'm sure 0:08:20.440 --> 0:08:23.280 people in corporation, certainly at Intel, when you go to 0:08:23.360 --> 0:08:26.240 a generative AI website, it says, hey, be careful what 0:08:26.280 --> 0:08:29.880 you enter into this textbox. And so moving this stuff 0:08:29.920 --> 0:08:32.680 offline gives you a couple of things. Yes, you get 0:08:32.840 --> 0:08:35.840 chain of custody over the data. It's private, right, it's 0:08:35.880 --> 0:08:40.840 working on your information offline, right, so it doesn't have 0:08:40.880 --> 0:08:43.160 to go to the cloud. That's a big one. Cost 0:08:43.280 --> 0:08:46.960 is another component. A lot of the most powerful AI 0:08:47.080 --> 0:08:50.559 services online, you know, ten to twenty bucks a month, 0:08:51.240 --> 0:08:55.400 and it's not cheap to have several of those. The 0:08:55.520 --> 0:09:02.760 last that is interesting is cloud servers are intrinsically pricey. 0:09:03.080 --> 0:09:06.800 I'm not saying they're expensive, but for AI as a 0:09:08.000 --> 0:09:12.360 genre of software to truly take off and thrive in 0:09:12.400 --> 0:09:15.600 all the way the software vendors want it to, it 0:09:15.679 --> 0:09:19.160 has to reach the local PC. It has to reach 0:09:19.640 --> 0:09:22.160 you know, tens of millions of users, hundreds of millions 0:09:22.200 --> 0:09:25.320 of users on a local device and that's sort of 0:09:25.320 --> 0:09:29.800 that critical mass install base that takes this effort to 0:09:29.880 --> 0:09:32.920 the next level. Cloud was sort of one point zero 0:09:32.920 --> 0:09:35.480 of AI, and now we're trying to, you know, for 0:09:35.720 --> 0:09:37.600 one of a better term, create the two point zero 0:09:37.600 --> 0:09:41.000 where lots of people have access to this and it's 0:09:41.000 --> 0:09:44.160 widely available and in a couple of years Intel alone 0:09:44.160 --> 0:09:47.679 we want to get one hundred million accelerators for AI 0:09:47.760 --> 0:09:48.680 into the hands of people. 0:09:49.280 --> 0:09:52.760 And do you have any examples of at this early 0:09:52.800 --> 0:09:57.080 stage of actually pushing the boundaries of using this sort 0:09:57.120 --> 0:10:00.920 of power in a local PC to do some really 0:10:00.920 --> 0:10:01.720 interesting work. 0:10:03.160 --> 0:10:06.520 One for an enterprise that we've been tinkering with is 0:10:07.200 --> 0:10:11.920 a technology called rag rag, and it's the idea where 0:10:12.040 --> 0:10:17.600 you have a language model running offline on the user's PC. 0:10:18.320 --> 0:10:24.679 But this RAG component can scan your documents, your corporate information, 0:10:24.960 --> 0:10:30.400 and then specialize the LM to be for you, your work, 0:10:30.520 --> 0:10:35.120 your knowledge. So just as an example, we scanned one 0:10:35.200 --> 0:10:38.920 state's DMV manual, which hundreds of pages long for the 0:10:38.920 --> 0:10:42.160 Department of Motor Vehicle manual and now you can ask 0:10:42.440 --> 0:10:47.400 very specific procedural and legal questions about the subject of 0:10:47.400 --> 0:10:51.439 that manual and it'll spit back highly accurate answers for you. 0:10:52.160 --> 0:10:56.400 And if you extend out out words, protecting and promoting 0:10:56.559 --> 0:11:00.520 institutional knowledge is hard. You might have that employ that's 0:11:00.520 --> 0:11:04.200 been there for twenty years and has all that institutional 0:11:04.240 --> 0:11:06.520 knowledge in their head and if they leave, it goes 0:11:06.559 --> 0:11:11.920 with them. But a RAG model could synthesize that knowledge 0:11:11.960 --> 0:11:14.920 for people, so a new employee could just ask a 0:11:15.000 --> 0:11:18.320 question in a text box and get an accurate answer 0:11:18.400 --> 0:11:21.600 about what that company's working on or what this feature does. 0:11:22.240 --> 0:11:25.120 And that's just huge for business. 0:11:25.480 --> 0:11:28.959 Yeah, you know, I'm from a small business sort of background. 0:11:29.120 --> 0:11:32.280 My dad has a small business. And the fact that 0:11:32.320 --> 0:11:35.120 you can bring this power to the small and micro 0:11:35.200 --> 0:11:39.280 businesses as well without having to pay these cloud based 0:11:40.120 --> 0:11:42.880 prices and also in conjunction with using some of the 0:11:42.960 --> 0:11:46.280 open source type software that I know Intel is very 0:11:46.280 --> 0:11:50.120 supportive of. I'm just really excited to see the little guys, 0:11:50.200 --> 0:11:52.320 you know, be able to compete with some of the 0:11:52.360 --> 0:11:54.040 technology that the big boys have. 0:11:54.840 --> 0:11:58.480 Absolutely, AI is, at the end of the day, a 0:11:58.520 --> 0:12:02.440 force multiplier for a person. Right like at the root, 0:12:02.679 --> 0:12:06.720 AI is designed to save time writing meeting minutes or 0:12:06.800 --> 0:12:11.440 email summaries or drafting in outline. These are all just 0:12:11.520 --> 0:12:15.360 like time consuming tasks for people, and they don't require 0:12:16.320 --> 0:12:19.960 skill per se, but it's it's time consuming, and so 0:12:20.040 --> 0:12:22.480 being able to offload that to a digital assistant that 0:12:22.480 --> 0:12:25.720 can just sort of ninety percent or ninety five percent 0:12:25.840 --> 0:12:29.040 do that for you allows you to refocus your efforts 0:12:29.040 --> 0:12:31.320 back to something else that is more productive and more 0:12:31.360 --> 0:12:34.880 worthy of your time. And especially for a small business, 0:12:35.120 --> 0:12:39.600 administrative overhead, bureaucratic overhead is hard, it's time consuming. I 0:12:39.640 --> 0:12:44.960 myself own a single member LLC and administrative stuff takes 0:12:45.040 --> 0:12:47.000 up a ton of my time. Like I'd love to 0:12:47.000 --> 0:12:48.079 outsource that to AI. 0:12:49.559 --> 0:12:53.000 That's right. In terms of Intel's history with past, you know, 0:12:53.040 --> 0:12:58.120 technological revolutions, I'm reminded of Intel's initiative to get Wi 0:12:58.120 --> 0:13:02.280 Fi into every your laptop and that was code named Centrino, 0:13:03.160 --> 0:13:08.960 and it's interesting to hear that again. Intel are trying 0:13:09.000 --> 0:13:13.000 to push new technology so that it's ubiquitous, and that's 0:13:13.040 --> 0:13:17.959 exactly what they're doing with these aipcs. And in ten 0:13:18.040 --> 0:13:22.000 twenty years time, I think that aipowered PCs will be 0:13:22.080 --> 0:13:26.000 so ubiquitous that we won't think anything of it. I'd 0:13:26.040 --> 0:13:30.160 like Robert your thoughts on that and more insights into 0:13:30.200 --> 0:13:34.200 the way Intel is evolving their strategy and Intel's role 0:13:34.240 --> 0:13:35.480 in this. Yeah. 0:13:35.520 --> 0:13:38.840 Actually, Centrino is a really nice analogy because that's sort 0:13:38.880 --> 0:13:42.160 of what we're trying to do with these AI accelerators. 0:13:42.440 --> 0:13:45.840 It's not a huge tweak to the configuration of a 0:13:45.880 --> 0:13:51.640 system design because most of the work happens inside the CPU, 0:13:51.760 --> 0:13:56.720 and there are very few external requirements that would change 0:13:56.760 --> 0:14:00.000 a system designed to make this possible. Right, So it's 0:13:59.720 --> 0:14:03.480 a system vendor could theoretically update last year's chassis to 0:14:03.520 --> 0:14:07.840 have a new CPU and that would confer the benefits 0:14:07.880 --> 0:14:12.320 from Core Ultra and an AI workloads and Centrino not 0:14:12.440 --> 0:14:15.079 much different, right. You're adding a Wi Fi chip and 0:14:15.200 --> 0:14:16.880 an antenna to the system. 0:14:17.080 --> 0:14:19.040 Yes, But for those. 0:14:18.960 --> 0:14:21.720 Of you who weren't around during the Centrino days, it 0:14:21.880 --> 0:14:24.400 used to be very common that a laptop would not 0:14:24.520 --> 0:14:28.640 have Wi Fi, which seems sort of unimaginable now, but 0:14:29.000 --> 0:14:32.320 that's because Intel made this massive effort to make it 0:14:32.360 --> 0:14:35.480 common and we all sort of take it for granted now. 0:14:36.000 --> 0:14:39.480 Another analogous moment for me is the addition of graphics 0:14:39.680 --> 0:14:43.000 in the processor. I was a hardware reviewer when that 0:14:43.200 --> 0:14:47.200 started happening, and I remember the conversations back then like 0:14:48.200 --> 0:14:50.360 why are we doing this? You can't even play a 0:14:50.360 --> 0:14:52.720 game on it, what's the point? This is just going 0:14:52.760 --> 0:14:56.280 to make CPUs more expensive, YadA, YadA, YadA. Today web 0:14:56.360 --> 0:15:00.880 pages are rendered by your graphics card orphics accelerator in 0:15:00.920 --> 0:15:04.920 the processor, your browser uses it. It's everywhere. So both of 0:15:04.960 --> 0:15:07.880 those are very foundational examples for me, because I see 0:15:08.000 --> 0:15:12.200 AI as being quite analogous in both respects. But I 0:15:12.240 --> 0:15:15.800 think history will bear out that this was a pivotal 0:15:15.840 --> 0:15:19.160 moment and AI will be very, very widespread, just like 0:15:19.200 --> 0:15:20.480 graphics and just like Wi Fi. 0:15:23.520 --> 0:15:26.640 Coming up next on Technically Speaking and Intel podcast. 0:15:28.440 --> 0:15:31.800 Being familiar with how prompts work, it's going to be 0:15:31.840 --> 0:15:34.240 a key business skill, and I think there'll be a 0:15:34.280 --> 0:15:38.000 real advantage for employees who know how to engineer a 0:15:38.040 --> 0:15:40.240 good prompt to get a great result quickly. 0:15:42.000 --> 0:15:44.080 We'll be right back after a brief message from our 0:15:44.080 --> 0:15:58.120 partners that Intel welcome back to Technically Speaking. I'm here 0:15:58.200 --> 0:16:06.400 now with Robert Hallett getting back to I guess more 0:16:06.440 --> 0:16:09.320 on the business side of things. Maybe if you can 0:16:09.360 --> 0:16:11.240 talk a little bit about our friends in the IT 0:16:11.520 --> 0:16:14.400 department having to manage the security and all these sort 0:16:14.440 --> 0:16:17.080 of things. Yeah, what's some of the benefits they could 0:16:17.120 --> 0:16:19.680 look forward to in terms of managing these types of 0:16:19.720 --> 0:16:21.800 aipcs in their enterprise. 0:16:22.480 --> 0:16:25.120 I think it'll make every ITDM happy that at the 0:16:25.200 --> 0:16:27.960 end of the day, these AI applications are nothing more 0:16:28.120 --> 0:16:32.280 than endpoint applications. You download them, you install them, and 0:16:32.320 --> 0:16:35.400 they're from software vendors, and I'm sure they'll be breakout 0:16:35.520 --> 0:16:38.680 ISVs that come under the scene as a result of 0:16:38.720 --> 0:16:43.200 this AI transformation, but largely speaking, trusted vendors doing new work. 0:16:43.760 --> 0:16:48.280 And because it's entirely offline, your user has custody of 0:16:48.360 --> 0:16:51.240 the data and the information, which is no different from 0:16:51.240 --> 0:16:54.600 any other application today. So it's not like this transformation 0:16:55.680 --> 0:16:58.960 comes part and parcel with like a radical transformation and 0:16:59.080 --> 0:17:02.200 endpoint management, which would make it way harder. From a 0:17:02.240 --> 0:17:07.960 security point of view, AI has some very interesting benefits 0:17:08.160 --> 0:17:11.120 for security models. So now this is the ITDM point 0:17:11.119 --> 0:17:15.800 of view. We recently with Dell and CrowdStrike, and if 0:17:15.800 --> 0:17:19.280 people don't know what CrowdStrike is, it's, amongst many things, 0:17:19.320 --> 0:17:24.280 an endpoint security solution which is specifically designed to help 0:17:24.840 --> 0:17:28.720 prevent threats that don't attack files. Many of the tax 0:17:29.000 --> 0:17:32.320 that go into a system now aren't like a virus 0:17:32.400 --> 0:17:36.000 that infects a file, it's actually resident in memory. They're 0:17:36.080 --> 0:17:43.040 fileless attacks, and these are way harder to detect and prevent. 0:17:43.600 --> 0:17:48.320 So CrowdStrike uses convolutional neural networks, which is a simpler 0:17:48.359 --> 0:17:52.560 form of neural network or AI, to monitor the real 0:17:52.560 --> 0:17:57.240 time conditions of the system and see if something unusual 0:17:57.600 --> 0:18:01.959 is happening. And this experiment moves these convolutional neural network 0:18:02.000 --> 0:18:06.040 models or CNNs to that neural processing unit or the 0:18:06.160 --> 0:18:09.040 NPU in coraltrum. It did a couple things when we 0:18:09.080 --> 0:18:13.879 did that. First, it gave the processor cores twenty percent 0:18:13.960 --> 0:18:17.119 performance back. The second thing it did it made the 0:18:17.160 --> 0:18:20.920 security model slightly smaller, so it had a smaller memory 0:18:20.920 --> 0:18:25.280 footprint now so user gets some RAM back. And it 0:18:25.400 --> 0:18:28.880 also improved the accuracy of the model because they can 0:18:28.920 --> 0:18:32.400 make the model computationally bigger, right because it has its 0:18:32.440 --> 0:18:36.800 own dedicated accelerator now to run on. And so security 0:18:36.840 --> 0:18:40.280 got better and crowdstreg is a very popular solution, and 0:18:40.359 --> 0:18:44.560 there are other solutions in the pipe for phishing detection, 0:18:44.840 --> 0:18:48.159 which is notoriously hard pattern matching problem. And that's just 0:18:48.200 --> 0:18:52.520 two examples of the way security can be enhanced by 0:18:52.640 --> 0:18:56.360 offloading to an AI specific accelerator. 0:18:56.880 --> 0:19:00.239 Yeah, and we talked a little bit about you know, 0:19:00.359 --> 0:19:04.880 businesses and enterprises adopting these new aipcs. Is anything special 0:19:04.920 --> 0:19:08.359 that IT departments and organizations need to do to prepare 0:19:08.359 --> 0:19:08.840 for this. 0:19:09.280 --> 0:19:12.480 I'll say there's probably three things that an ITDM would 0:19:12.520 --> 0:19:16.080 want to think about if they intend to use large 0:19:16.119 --> 0:19:21.280 language models or just generative AI. Those workloads are pretty 0:19:21.320 --> 0:19:25.560 sensitive to memory bandwidth, so you wouldn't normally think about 0:19:25.640 --> 0:19:30.320 memory bandwidth in a system purchase, but making sure that 0:19:30.480 --> 0:19:34.240 it has two memory sticks over one. For example, right, 0:19:34.280 --> 0:19:36.120 if you want to get sixteen gigs a RAM, make 0:19:36.119 --> 0:19:38.320 sure that's two by eight instead of one by sixteen, 0:19:38.640 --> 0:19:42.639 Just as an example, that will dramatically improve the performance 0:19:42.840 --> 0:19:47.919 of the LLM, because these language models are fundamentally limited 0:19:48.240 --> 0:19:52.760 or enhanced by the performance of the memory subsystem. Outside 0:19:52.800 --> 0:19:57.840 of that, for other forms of AI in the year ahead, 0:19:58.680 --> 0:20:03.080 there is a calculation called TOPS or terra operations per second, 0:20:03.080 --> 0:20:06.240 which is sort of a ballpark for how much AI 0:20:06.400 --> 0:20:12.280 performance a device can give you. Software makes or breaks 0:20:12.440 --> 0:20:15.960 the acquisition of that TOPS figure. So I can give 0:20:16.000 --> 0:20:19.720 you a billion TOPS, and if I had a very 0:20:19.760 --> 0:20:23.960 poor software stack under that you would never see the 0:20:24.040 --> 0:20:28.160 billion tops. So there's a new level of knowledge the 0:20:28.200 --> 0:20:32.600 ITDM needs to develop on sort of software stack robustness 0:20:33.560 --> 0:20:37.000 underneath that rating. Okay, so you have to be familiar 0:20:37.080 --> 0:20:39.520 with what Intel's doing in the software space, what its 0:20:39.520 --> 0:20:42.960 competitors are doing in the software space to really understand 0:20:43.640 --> 0:20:45.920 whether or not you're going to get a good experience 0:20:46.280 --> 0:20:48.440 out of the device you're purchasing. So that's number two, 0:20:49.160 --> 0:20:52.600 and number three I think would be to say that 0:20:53.160 --> 0:20:56.600 itdms will probably be faced with a lot of advocacy 0:20:56.800 --> 0:21:01.879 for the NPU as an AIX, but it's important to 0:21:01.960 --> 0:21:07.160 understand that the software industry broadly also wants to use 0:21:07.520 --> 0:21:11.080 GPU and CPU as well. So if you make an 0:21:11.160 --> 0:21:15.959 upgrade decision that is all in on NPU performance and 0:21:16.000 --> 0:21:20.159 you didn't check on the GPU or the CPU, you 0:21:20.280 --> 0:21:22.920 may be out in the cold on performance or power 0:21:22.960 --> 0:21:26.760 efficiency for these Frankly a large number of workloads that 0:21:26.880 --> 0:21:30.399 use graphics and CPU for AI acceleration. Those are the 0:21:30.440 --> 0:21:33.400 three things that I would say are new or different 0:21:33.920 --> 0:21:38.480 in this era, But overall it is something that you 0:21:38.520 --> 0:21:43.119 can integrate into your upgrade cycle piecemeal. YEP, Newer devices 0:21:43.200 --> 0:21:46.200 will be faster. I mean, they always are, but it's 0:21:46.200 --> 0:21:49.480 not like you're missing out on a new feature, right, 0:21:49.520 --> 0:21:51.959 We're going to make them faster, But the features are 0:21:51.960 --> 0:21:54.600 delivered by the software, are not the hardware. You can 0:21:54.680 --> 0:21:56.520 kind of get in at any point and get the 0:21:56.560 --> 0:21:59.120 goodness of AI, which is pretty cool as well. 0:21:59.800 --> 0:22:04.119 Yeah, yeah, And are you helping the software vendors, I 0:22:04.200 --> 0:22:07.400 guess compile their code so that it will help them 0:22:07.760 --> 0:22:09.879 really utilize that hardware underneath. 0:22:10.840 --> 0:22:13.760 Yeah, that's the secret of AI. And I'll start with 0:22:13.800 --> 0:22:16.600 an analogy of PC gaming, which I think is a 0:22:16.640 --> 0:22:19.679 lot more familiar to people. So at the bottom, you 0:22:19.760 --> 0:22:23.480 have a piece of hardware, a graphics card, and they 0:22:23.520 --> 0:22:27.480 tell you, you know, it's certain terra flops or gigaflops of performance, 0:22:28.000 --> 0:22:32.679 but everybody knows that really depends on how well optimized 0:22:32.720 --> 0:22:37.000 the game engine is, how well optimized the game you're 0:22:37.080 --> 0:22:41.520 running is, how good the graphics drivers are. AI is 0:22:42.000 --> 0:22:47.000 no different. AI accelerators are actually exposed in direct X 0:22:47.160 --> 0:22:52.119 in Windows as a GPU without display outputs, so the 0:22:52.160 --> 0:22:57.280 system sees them as essentially graphics cards, and instead of 0:22:57.320 --> 0:23:01.080 game engines you have AI models will have features and 0:23:01.320 --> 0:23:05.080 apps just like games you even have run times or 0:23:05.080 --> 0:23:08.439 an environment where the code is running in there's a 0:23:08.480 --> 0:23:12.240 DirectX run time for graphics, there is equivalent run times 0:23:12.280 --> 0:23:16.280 for AI. So in many respects, the AI software stack 0:23:16.480 --> 0:23:20.959 looks and works a lot like the gaming software stack. 0:23:21.400 --> 0:23:25.440 And so if people think about all the times that 0:23:25.760 --> 0:23:29.680 a GPU that's supposed to be faster on paper didn't 0:23:29.760 --> 0:23:32.679 live up to that number because of one software reason 0:23:32.800 --> 0:23:36.960 or another, that is a possible reality for AI as well. 0:23:37.000 --> 0:23:40.800 And that's why it's so important to make your decision 0:23:41.720 --> 0:23:45.280 not just on tops or what the accelerator is, but 0:23:45.960 --> 0:23:49.000 on the robustness of the software underneath, because that's where 0:23:49.000 --> 0:23:50.160 it really happens. 0:23:52.240 --> 0:23:55.639 Regular listeners to the show Mike remember back in season one, 0:23:55.720 --> 0:23:58.520 we divided a whole episode on the skills that workers 0:23:58.520 --> 0:24:02.240 will need in order to take advantage of the kinds 0:24:02.240 --> 0:24:05.919 of AI tools we discussed today. AI technology can only 0:24:06.080 --> 0:24:09.840 expand our expectations of what's possible if we understand the 0:24:09.880 --> 0:24:12.760 most effective ways to use it. So I asked Robert 0:24:12.800 --> 0:24:15.119 for his thoughts and what workers should focus on to 0:24:15.160 --> 0:24:17.960 take advantage of this new wave of technology, and he 0:24:18.040 --> 0:24:20.480 began his answer with something just about everyone does on 0:24:20.520 --> 0:24:23.200 the Internet, every day. 0:24:23.320 --> 0:24:26.000 Ooh, that's a good one. I actually want to start 0:24:26.080 --> 0:24:29.800 briefly at web searching. You know, web searching, a good, 0:24:30.080 --> 0:24:34.680 well composed query is a learned skill. We have all 0:24:34.800 --> 0:24:38.800 encountered people that haven't quite learned that skill, and they 0:24:38.840 --> 0:24:41.639 get bad search results and they're frustrated with a search engine. 0:24:41.760 --> 0:24:44.399 And not a lot of people teach that skill because 0:24:44.400 --> 0:24:48.840 it's its own language of sorts. You know, you want 0:24:48.880 --> 0:24:51.720 to freeze things to a search box differently than how 0:24:51.760 --> 0:24:54.639 I would ask it out loud. So that takes me 0:24:54.680 --> 0:24:59.960 to AI, where you are still engineering a search of sorts. 0:25:00.640 --> 0:25:03.880 It's called a prompt now, but it's a search and 0:25:04.760 --> 0:25:07.360 the skill is in like, how do you craft that 0:25:07.600 --> 0:25:11.399 prompt to get the result that you're looking for? You 0:25:11.440 --> 0:25:15.840 have to be somewhat specific, and you have to understand 0:25:15.880 --> 0:25:18.480 the limitations of the AI model you're working with. So 0:25:18.720 --> 0:25:22.560 let's take an image creation model. Some of them only 0:25:22.640 --> 0:25:26.280 support what's called positive prompts. You have to describe the 0:25:26.280 --> 0:25:29.600