WEBVTT - Smart Talks with IBM: The power of Granite in business 0:00:04.440 --> 0:00:12.639 Welcome to tech Stuff, a production from iHeartRadio. This season 0:00:12.720 --> 0:00:16.200 on smart Talks with IBM, Malcolm Gladwell and team are 0:00:16.239 --> 0:00:19.479 diving into the transformative world of artificial intelligence with a 0:00:19.520 --> 0:00:24.120 fresh perspective on the concept of open What does open 0:00:24.560 --> 0:00:27.920 really mean in the context of AI. It can mean 0:00:28.000 --> 0:00:31.600 open source code or open data, but it also encompasses 0:00:31.720 --> 0:00:36.040 fostering an ecosystem of ideas, ensuring diverse perspectives are heard, 0:00:36.159 --> 0:00:39.919 and enabling new levels of transparency. Join hosts from your 0:00:39.960 --> 0:00:43.560 favorite Pushkin podcasts as they explore how openness and AI 0:00:43.800 --> 0:00:49.040 is reshaping industries, driving innovation, and redefining what's possible. You'll 0:00:49.080 --> 0:00:52.360 hear from industry experts and leaders about the implications and 0:00:52.400 --> 0:00:56.160 possibilities of open AI, and of course, Malcolm Gladwell will 0:00:56.200 --> 0:00:58.200 be there to guide you through the season with his 0:00:58.360 --> 0:01:01.840 unique insights. Look out for new episodes of Smart Talks 0:01:01.920 --> 0:01:05.360 every other week on the iHeartRadio app, Apple Podcasts, or 0:01:05.360 --> 0:01:08.880 wherever you get your podcasts, and learn more at IBM 0:01:09.080 --> 0:01:20.920 dot com, slash smart Talks. 0:01:18.440 --> 0:01:26.280 Pushkin Hello, Hello, Welcome to Smart Talks with IBM, a 0:01:26.319 --> 0:01:31.640 podcast from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Glawell. 0:01:32.280 --> 0:01:35.800 This season, we're diving back into the world of artificial intelligence, 0:01:36.120 --> 0:01:39.119 but with a focus on the powerful concept of open 0:01:39.640 --> 0:01:45.119 its possibilities, implications, and misconceptions. We'll look at openness from 0:01:45.200 --> 0:01:47.720 a variety of angles and explore how the concept is 0:01:47.760 --> 0:01:52.080 already reshaping industries, ways of doing business and our very 0:01:52.120 --> 0:01:57.280 notion of what's possible. In today's episode, Jacob Goldstein sat 0:01:57.320 --> 0:02:01.440 down with Mariam Ashuri, the Director of Product Management and 0:02:01.480 --> 0:02:05.240 a Head of product for IBM's Watson x dot AI, 0:02:05.760 --> 0:02:09.680 where she spearheads the product strategy and delivery of IBM's 0:02:09.720 --> 0:02:14.400 watsonx foundation models. She is a technologist with more than 0:02:14.440 --> 0:02:19.960 fifteen years of experience developing data driven technologies. The conversation 0:02:20.080 --> 0:02:24.200 focused on how enterprises can use technology to build and 0:02:24.280 --> 0:02:30.240 deliver greater transparency in AI. With Granite, Mariam explained how 0:02:30.240 --> 0:02:34.640 Grantite can be utilized to improve efficiency across various domains. 0:02:35.280 --> 0:02:38.560 She discussed how these models are being used in real 0:02:38.600 --> 0:02:43.800 world business applications, particularly in areas like customer care, where 0:02:43.840 --> 0:02:48.920 AI can help enable quick, accurate responses based on internal 0:02:49.160 --> 0:02:54.280 company data. Mariam provided a fascinating look into how enterprises 0:02:54.320 --> 0:02:59.919 have moved from mere experimentation with generative AI to actual production, 0:03:01.000 --> 0:03:05.880 navigating challenges such as increased latency, cost and energy consumption. 0:03:06.880 --> 0:03:11.160 She highlighted how the emerging trend of smaller models customized 0:03:11.200 --> 0:03:15.520 with proprietary data can potentially deliver high performance at a 0:03:15.560 --> 0:03:19.440 fraction of the cost, marking a significant shift in how 0:03:19.680 --> 0:03:24.160 enterprises leverage AI. Whether you're an AI enthusiast or a 0:03:24.160 --> 0:03:27.640 business leader looking to harness the power of artificial intelligence, 0:03:28.200 --> 0:03:34.200 this episode is packed with valuable insights and forward thinking strategies. 0:03:38.520 --> 0:03:40.480 Let's just start with your background. How did you come 0:03:40.520 --> 0:03:41.640 to work at IBM. 0:03:42.360 --> 0:03:45.840 I join IBM right after I graduated. I have an 0:03:45.840 --> 0:03:51.920 AI background, and throughout the years, I've held many roles 0:03:51.960 --> 0:03:57.440 in design, engineering, development research, mostly focused on AI application 0:03:57.560 --> 0:04:01.680 development and design. In my current job, I'm the product 0:04:01.680 --> 0:04:04.840 owner for What's the Next Day I, which is the 0:04:05.800 --> 0:04:10.280 IBM platform for enterprise AI. What excites me about this job, 0:04:10.320 --> 0:04:14.120 I would say, is the technology advancements over the last 0:04:14.120 --> 0:04:17.320 eighteen months in the market. We've been witnessing how generative 0:04:17.360 --> 0:04:20.040 ALI has been changing the market. The way that I 0:04:20.120 --> 0:04:23.440 see that is JENNAI has been perhaps one of the 0:04:23.560 --> 0:04:27.360 largest paradigm shifts. When we think about productivity the same 0:04:27.400 --> 0:04:32.800 way that Internet and personal computers impacted the productivity of workforce. 0:04:32.960 --> 0:04:37.840 Now we are witnessing another wave of all those opportunities 0:04:37.880 --> 0:04:41.120 that it can unlock for especially enterprise AI when it 0:04:41.160 --> 0:04:45.240 comes to enhancing the productivity of the workforce and releasing 0:04:45.320 --> 0:04:49.960 some time that can potentially be put into creating more 0:04:50.080 --> 0:04:54.480 value work for enterprise. So that's the major part that 0:04:54.640 --> 0:04:58.480 I picked this team to have an impact on the 0:04:58.520 --> 0:05:02.560 market and the community, but also of course using the 0:05:03.160 --> 0:05:06.080 skills that I gain through all these years through IBM 0:05:06.279 --> 0:05:10.240 to help to establish IBM as the market leader for 0:05:10.400 --> 0:05:11.159 enterprise AI. 0:05:11.720 --> 0:05:15.800 So you talked about jenai as this sort of generational, 0:05:15.960 --> 0:05:21.000 transformational technological force, and I'm curious just in terms of 0:05:21.440 --> 0:05:23.920 how it's going to come into the world, Like, how 0:05:23.920 --> 0:05:28.000 do you see market adoption of genai sort of evolving 0:05:28.040 --> 0:05:28.520 from here? 0:05:29.320 --> 0:05:32.880 Well, last year was the year of excitement about generative AI. 0:05:33.040 --> 0:05:36.159 Most of the companies were experimenting and exploring with GENI. 0:05:36.880 --> 0:05:40.440 We see that energy shifted towards how to best monetize 0:05:40.480 --> 0:05:43.360 that technology. Almost half of the market has moved from 0:05:43.640 --> 0:05:48.839 investigation to pilots, ten percent has moved to production. When 0:05:48.880 --> 0:05:52.520 you're exploring with this technology, you're looking for a valve factor, 0:05:52.800 --> 0:05:56.320 You're looking for an AHA moment. That's why very large 0:05:56.320 --> 0:06:00.919 general purpose models shine. But as companies move toward production 0:06:01.040 --> 0:06:03.960 and scale, they soon realize the past success is not 0:06:04.000 --> 0:06:09.039 that straightforward. For example, they're larger the model, the larger 0:06:09.120 --> 0:06:13.680 computer resources it requires. That translates to increased latency that's 0:06:13.680 --> 0:06:18.080 your response time. That translates to increased cost. That translates 0:06:18.120 --> 0:06:21.680 to increase carbon food print, and energy consumption. So think 0:06:21.720 --> 0:06:25.360 about that. At the scale of enterprise in production, some 0:06:25.440 --> 0:06:28.880 of them can be a showstopper. Because of this reason, 0:06:29.160 --> 0:06:33.640 what actually c is emerging in the market is instead 0:06:33.680 --> 0:06:39.520 of focusing on very large general purpose models, coming back 0:06:39.600 --> 0:06:45.280 to very small, trustworthy models that they can customize on 0:06:45.360 --> 0:06:48.920 their own proprietary data. That's the data about their customers, 0:06:48.960 --> 0:06:52.719 that the data about their specific domains to create something 0:06:52.800 --> 0:06:57.400 differentiated that is much smaller and delivers the performance that 0:06:57.440 --> 0:07:00.760 they want on a target use is for a fraction 0:07:00.839 --> 0:07:01.360 of the cost. 0:07:01.760 --> 0:07:05.640 Uh huh. So let's talk a little bit more specifically 0:07:05.680 --> 0:07:09.800 about what you're working on. Let's talk about granite. First 0:07:09.800 --> 0:07:11.480 of all, tell me what is granite. 0:07:12.040 --> 0:07:17.560 Granite is our industrial leading family of models, flagship IBM models. 0:07:18.320 --> 0:07:22.160 These are the models that we train from scratch. When 0:07:22.240 --> 0:07:25.640 offered to our platform, we offer indemnification and we stand 0:07:25.640 --> 0:07:31.360 behind them today. It comes in four flavors, language, code, 0:07:32.040 --> 0:07:39.520 time series and geospecial models. Granite language series is covering English, Spanish, German, 0:07:39.960 --> 0:07:44.800 Portuguese and Japanese. We have a combination of commercial and 0:07:45.000 --> 0:07:48.960 open source language models on Granite. For example, we recently 0:07:49.120 --> 0:07:54.280 released the Granite seven B language model small powerful English model. 0:07:55.040 --> 0:07:58.360 On the code front, our models are state of the 0:07:58.480 --> 0:08:01.640 art models ranging from from three billion to thirty four 0:08:01.680 --> 0:08:07.560 billion parameters. These are very powerful models that performs or 0:08:07.680 --> 0:08:11.560 outperforms in some cases the popular open source models in 0:08:11.600 --> 0:08:13.840 their weight class. So very powerful models. 0:08:14.040 --> 0:08:16.720 So I get the idea a big picture about these models, 0:08:16.760 --> 0:08:18.440 but it would be helpful to just get a sense 0:08:18.480 --> 0:08:20.600 specifically of what they're doing, Like, can you give me 0:08:20.640 --> 0:08:24.280 any specific examples of how these models are being used 0:08:25.080 --> 0:08:27.360 in businesses in the real world right now? 0:08:28.520 --> 0:08:31.800 Well, the top use cases for generative AI are really 0:08:31.880 --> 0:08:38.760 content generation, summarization, information extraction. Perhaps the most popular use 0:08:38.800 --> 0:08:42.480 case that we are seeing in enterprise is content grounded 0:08:42.559 --> 0:08:46.800 question and answering, So using these models as a base 0:08:47.080 --> 0:08:49.960 to connect them to a body of information let's say 0:08:50.000 --> 0:08:54.320 their policies, their documents that is internal to the enterprise, 0:08:54.600 --> 0:08:58.880 and get the model to provide answers based on that questions. 0:08:59.200 --> 0:09:03.160 One example of that is for customer agents customer care, 0:09:03.559 --> 0:09:08.120 when a customer is asking a question. Previously, the agent 0:09:08.200 --> 0:09:11.720 that responds to the customer had to answer the question 0:09:11.840 --> 0:09:14.600 and if they don't know the answer escalated to the product, 0:09:14.760 --> 0:09:18.559 especially keeping people on hold on the line to go 0:09:18.640 --> 0:09:21.360 figure out the answer for that and then come back. 0:09:21.440 --> 0:09:23.800 You can think of the time it takes to resolve 0:09:23.840 --> 0:09:27.520 an issue. But now with LLMS, we have an opportunity 0:09:27.559 --> 0:09:31.600 to automatically retrieve the information based on the internal documents 0:09:31.640 --> 0:09:34.640 of the company, formulate an answer, show it to the 0:09:34.720 --> 0:09:38.080 human agent, and then if they verify with the sources 0:09:38.080 --> 0:09:40.800 of varies coming from, they can just translate it directly 0:09:40.840 --> 0:09:44.959 to the customer. This is a very simple example of 0:09:45.000 --> 0:09:46.840 how it's impacting the customer care. 0:09:47.320 --> 0:09:51.800 So one big theme of this season is this idea 0:09:51.800 --> 0:09:55.200 of open And one of the things that's interesting to 0:09:55.320 --> 0:09:59.120 me about the work you're doing is you are using 0:09:59.200 --> 0:10:02.440 not only ran at this model IBM developed, but you're 0:10:02.520 --> 0:10:06.360 also using third party models right from other places. So 0:10:06.400 --> 0:10:08.520 tell me about that work and how that is sort 0:10:08.559 --> 0:10:11.840 of fitting into your kind of real world typically enterprise 0:10:11.960 --> 0:10:12.760 GENAI work. 0:10:13.400 --> 0:10:16.840 When it comes to model strategy, our strategy is really 0:10:16.880 --> 0:10:21.199 focused on two pillars, multimodel and multi deployment. It means 0:10:21.240 --> 0:10:24.280 that we don't believe one single model rules all the 0:10:24.400 --> 0:10:27.000 use cases. And I think at this point the market 0:10:27.000 --> 0:10:30.320 has also realized the enterprise markets in average today are 0:10:30.400 --> 0:10:34.680 using five to ten different models for different use cases. 0:10:34.880 --> 0:10:35.840 Oh interesting. 0:10:36.160 --> 0:10:38.439 So in our portfolio, if you look into what's on 0:10:38.480 --> 0:10:42.280 extradai today, we are offering a large sets of high performing, 0:10:42.400 --> 0:10:45.880 state of the art models coming from open source commercial 0:10:45.960 --> 0:10:49.480 models that we are bringing through our partners and also 0:10:49.559 --> 0:10:53.160 IBM developed models. In addition to all of these, we 0:10:53.280 --> 0:10:56.600 also have an option for bring your own model from 0:10:56.679 --> 0:10:59.760 outside the platform. Let's say you have a custom model 0:11:00.160 --> 0:11:02.200 that you made it yourself, you can bring it to 0:11:02.240 --> 0:11:07.320 the platform and really helping the customers to navigate through 0:11:07.440 --> 0:11:11.040 avoid range of models and pick the right model for 0:11:11.120 --> 0:11:15.040 their target use case. Throughout that, we've been heavily working 0:11:15.160 --> 0:11:18.200 with our partners and you know, this is the market 0:11:18.200 --> 0:11:21.440 that is evolving rapidly. We've been at the forefront of 0:11:21.440 --> 0:11:23.680 a spit to delivery. One example that I like to 0:11:23.960 --> 0:11:29.040 highlight is recently Metal released Lama four or five billion, 0:11:29.360 --> 0:11:31.920 such a powerful model. On the same day that it 0:11:32.080 --> 0:11:35.040 was released to the market, we made it available in 0:11:35.120 --> 0:11:38.160 our platform to our customers the same day, and not 0:11:38.240 --> 0:11:40.679 only we delivered it on the same day. We are 0:11:40.679 --> 0:11:45.160 offering competitive pricing but also flexibility in where to deploy. 0:11:45.280 --> 0:11:48.199 So we are giving an option to enterprise to deploy 0:11:48.280 --> 0:11:52.520 these models on the platform of dage choice, either multi 0:11:52.520 --> 0:11:56.400 cloud it can be gcpaws as youre IBM cloud, or 0:11:56.440 --> 0:12:02.320 on premises the same for mistrall Ai MISTROLEI recently released 0:12:02.320 --> 0:12:05.320 the model Misrol large two on the same day we 0:12:05.440 --> 0:12:08.360 delivered that through the platform. That's an example of a 0:12:08.400 --> 0:12:13.080 commercial model. Lama as open source, but large two is 0:12:13.200 --> 0:12:17.280 a commercial model that we made available through the platform Great. 0:12:18.080 --> 0:12:22.559 So I want to talk about enterprise grade foundation models, 0:12:23.280 --> 0:12:26.200 just to get into it briefly. What's a foundation model. 0:12:26.640 --> 0:12:30.319 People associate foundation models with a large language model, but 0:12:30.480 --> 0:12:33.800 large language models are really a subset of foundation models. 0:12:33.960 --> 0:12:37.880 Large language models are focused on language, but foundation models 0:12:37.920 --> 0:12:41.760 can be code generators, can be focused on time series 0:12:41.760 --> 0:12:44.360 model we talked about, they can be images, it can 0:12:44.400 --> 0:12:48.840 be jewy special models. So foundation model, as the term 0:12:48.960 --> 0:12:54.079 suggests that your foundations to create a series of subsequent 0:12:54.280 --> 0:12:58.640 models that can be customized for a downstream use case. 0:12:58.679 --> 0:13:02.079 And that's why they are calling that foundation models. LM 0:13:02.240 --> 0:13:04.240 is a good example of that as a subset for 0:13:04.400 --> 0:13:08.480 language that you can further customize on your specific data 0:13:08.880 --> 0:13:12.439 to get the model to do other works. So the 0:13:12.480 --> 0:13:16.000 core of these foundation models, they are basically trained on 0:13:16.040 --> 0:13:20.320 an ab third amount of data data sets that most 0:13:20.320 --> 0:13:23.400 of the institutions today are sourcing them from the internet, 0:13:23.559 --> 0:13:26.560 So you can imagine what can potentially go to those models, 0:13:26.600 --> 0:13:29.839 and then it comes to the enterprise and they start 0:13:29.920 --> 0:13:33.840 using it. So for us also, when we started looking 0:13:33.920 --> 0:13:38.800 into in particular, it was triggered by customers asking us 0:13:38.800 --> 0:13:42.280 to provide client protections on these models, and we started 0:13:42.280 --> 0:13:45.679 thinking about, let's look into how the models are trained 0:13:45.679 --> 0:13:49.200 and if you are comfortable of fering client protections on 0:13:49.440 --> 0:13:51.920 the models that are available in the market. And guess what, 0:13:52.400 --> 0:13:55.480 For a majority of these models, there is absolutely no 0:13:55.640 --> 0:13:59.520 visibility into what data vent into those models, not much 0:13:59.559 --> 0:14:03.840 transparent into how the model trains and the responsibility lies 0:14:03.880 --> 0:14:06.880 on you as the customers to start using those models. 0:14:06.920 --> 0:14:10.720 So just to be clear, that is presenting like potential risk, 0:14:10.880 --> 0:14:14.239 real potential risk to a company that is using these models. 0:14:14.360 --> 0:14:17.360 It is. It is a potential risk in particular for 0:14:17.480 --> 0:14:22.080 the customers in highly regulated industries. So what we did 0:14:22.160 --> 0:14:26.240 for Granite was when we started training these models from scratch, 0:14:26.320 --> 0:14:29.360 basically we went to the corpus of data that was 0:14:29.400 --> 0:14:32.920 available to us. So, for example, the very first version 0:14:33.000 --> 0:14:37.160 of Granite was exposed to twenty persons of its data 0:14:37.240 --> 0:14:40.000 from finance and legal because we have a lot of 0:14:40.760 --> 0:14:45.280 financial institutions as our clients. We worked directly with our 0:14:45.360 --> 0:14:50.480 IBM research to identify detectors for harmful information like haytyp 0:14:50.600 --> 0:14:52.239 use and profanity detectors. 0:14:52.800 --> 0:14:55.120 Okay, so we're talking about Granted, we're talking about this 0:14:55.320 --> 0:14:58.640 set of models IBM has developed. Let's talk about using 0:14:58.680 --> 0:15:03.680 Granite on WATSONEX compared to downloading open source models, Like, 0:15:03.720 --> 0:15:04.520 how do those differ? 0:15:05.160 --> 0:15:08.800 By using Granite and whats on X, you get two things. 0:15:09.160 --> 0:15:12.720 The first one is the client protection and in thementification 0:15:12.840 --> 0:15:15.040 that we talked about, you get that if the model 0:15:15.080 --> 0:15:18.840 is consumed through our platform and the second one is 0:15:19.080 --> 0:15:22.880 really the ecosystem of platform capabilities that we are offering 0:15:23.000 --> 0:15:26.200 to help you create value on top of those data. 0:15:26.280 --> 0:15:30.240 So for example, bringing your data to customize granted for 0:15:30.320 --> 0:15:33.640 your own specific use case. But also one thing that 0:15:33.680 --> 0:15:37.160 I like to highlight in particular is the AI governance. 0:15:37.720 --> 0:15:40.400 So when you get one of these pre train models, 0:15:40.640 --> 0:15:43.800 you put it in front of your own users. Through 0:15:43.840 --> 0:15:48.480 the input and instructions that the user provides for the model, 0:15:48.960 --> 0:15:53.200 they can notdge the model to potentially create undesired behavior 0:15:53.520 --> 0:15:56.200 and change the behavior of the model. And because of 0:15:56.240 --> 0:16:00.400 this is extremely important to automatically document the lead age 0:16:00.440 --> 0:16:04.320 of who touched the model at what point, so if 0:16:04.320 --> 0:16:06.760 something happens, you can trace it back and see where 0:16:06.800 --> 0:16:10.760 it's coming from. And that's what's an extra governance is 0:16:10.800 --> 0:16:15.360 offering automatically documenting the lineage. When you use the granite 0:16:15.520 --> 0:16:18.120 within the platform you get all of those, you can 0:16:18.360 --> 0:16:21.760 have the end to end governance, You can have access 0:16:21.800 --> 0:16:25.840 to all these scalable deployment opportunities that is available for you, 0:16:25.920 --> 0:16:28.800 like to allow you deploy them on the platform of 0:16:28.840 --> 0:16:32.320 your choice that we talked about, either multiple cloud or 0:16:32.600 --> 0:16:35.760 on prem and it also helps you to have access 0:16:35.800 --> 0:16:40.440 to avoid range of model customizations, approaches, prompt tuning, fine tuning, 0:16:40.680 --> 0:16:44.000 retrival augmented generations agents. There is a series of them 0:16:44.120 --> 0:16:46.600 available to use an apply to your model. 0:16:47.400 --> 0:16:51.880 This distinction between large language models and foundation models is 0:16:51.920 --> 0:16:56.400 eye opening. Mariam emphasized that foundation models can be tailored 0:16:56.400 --> 0:17:01.400 to specific tasks, but with that versatility comes a significant 0:17:01.480 --> 0:17:05.880 challenge the lack of transparency and how these models are trained. 0:17:06.680 --> 0:17:10.679 This composed a real risk, especially in highly regulated industries 0:17:11.000 --> 0:17:16.440 like finance. Essentially, by using Granite and watsonex together, enterprises 0:17:16.480 --> 0:17:19.200 get powerful and customizable tools. 0:17:20.400 --> 0:17:22.600 So let's talk about the future a little bit. What 0:17:22.680 --> 0:17:24.720 do you think are some of the big developments we're 0:17:24.840 --> 0:17:27.520 likely to see in the realm of AI models? 0:17:28.040 --> 0:17:32.359 Very good question. I feel like the generative AI of 0:17:32.440 --> 0:17:37.560 the past was powered by large language models. The generative 0:17:37.640 --> 0:17:41.919 AI of the future is going to reason, plan, act 0:17:42.280 --> 0:17:43.080 and reflect. 0:17:43.600 --> 0:17:47.000 Huh, and so I mean in the context of Granite 0:17:47.200 --> 0:17:50.639 in particular, like, what are we likely to see both 0:17:50.800 --> 0:17:52.680 you know, in the near term and in the sort 0:17:52.720 --> 0:17:54.000 of medium to long term. 0:17:54.560 --> 0:17:58.879 There are multiple elements to implement an agentic workflow that 0:17:58.920 --> 0:18:02.440 I just mentioned. One element of that is the LLM 0:18:02.520 --> 0:18:06.639 itself to be able to do the planning and reasoning 0:18:06.720 --> 0:18:11.080 and acting and doing something that we call tool calling. 0:18:11.480 --> 0:18:15.080 So basically, a series of tools are available to the model. 0:18:15.640 --> 0:18:18.520 You ask the model to call those and make a call. 0:18:18.680 --> 0:18:21.840 For example, we can say, hey, Granted, what is the 0:18:21.840 --> 0:18:27.600 weather like where Jacob lives. It's connect to web search API, 0:18:28.160 --> 0:18:30.919 look up your location. Then it's going to connect to 0:18:31.359 --> 0:18:35.720 weather API, calculate the weather and come back and formulate 0:18:35.760 --> 0:18:39.320 an answer and respond to that. So during this process, 0:18:39.880 --> 0:18:42.359 it first has to plan the task of how to 0:18:42.400 --> 0:18:45.280 answer that question, look into what are the tools that 0:18:45.320 --> 0:18:48.000 are available to it and call them, and that's an 0:18:48.000 --> 0:18:50.680 ability of the model to do that. What we did 0:18:50.720 --> 0:18:54.800 with Granted was we expanded the granite capabilities to be 0:18:54.880 --> 0:18:58.520 able to do function calling. So for example, today we 0:18:58.880 --> 0:19:01.960 have an open source granted to an eb function calling 0:19:02.040 --> 0:19:05.000 that is available on hugging face to try on and 0:19:05.040 --> 0:19:07.640 you can grab the model and the model has capability 0:19:07.680 --> 0:19:11.000 to do the tool callings. I'm anticipating that in the 0:19:11.040 --> 0:19:15.280 near future the planning and reasoning and acting and reflecting 0:19:15.320 --> 0:19:18.400 capabilities of the large language models are going to continue 0:19:18.440 --> 0:19:18.919 to evolve. 0:19:20.320 --> 0:19:24.359 So thinking now from the point of view of buyers 0:19:24.400 --> 0:19:28.040 and users of AIS really people who are listening from 0:19:28.040 --> 0:19:34.480 that perspective. As people are evaluating AI tools and solutions, 0:19:35.119 --> 0:19:38.000 what is the most important thing they should be thinking about? 0:19:38.080 --> 0:19:40.520 How do you think about kind of that process? 0:19:41.560 --> 0:19:44.880 I think they should always start with the area at 0:19:44.960 --> 0:19:49.040 which they think it would benefit from AI, and then 0:19:49.359 --> 0:19:53.360 within that area, look into what data they have available 0:19:53.520 --> 0:19:57.720 to potentially fit into those AI service architects do they 0:19:57.720 --> 0:20:01.280 have access to quality data? And the second question that 0:20:01.320 --> 0:20:03.199 they have to ask themselves is do I have a 0:20:03.240 --> 0:20:07.199 trusted partner that can supply what I need to be 0:20:07.240 --> 0:20:10.959 able to implement AI. That can be a collection of 0:20:11.000 --> 0:20:13.560 the foundation models that you're going to need, That can 0:20:13.600 --> 0:20:17.640 be a collection of the platform capabilities that the trusted 0:20:17.680 --> 0:20:21.040 partner can offer you to implement such a thing. The 0:20:21.119 --> 0:20:26.240 third thing is go and evaluate the regulations. Does regulation 0:20:26.600 --> 0:20:30.880 allow you to applyoy AI to the specific area that 0:20:31.400 --> 0:20:34.800 you are investigating and you're targeting for AI. And the 0:20:34.920 --> 0:20:38.160 last part, but not least, is back to the principles 0:20:38.200 --> 0:20:41.840 of design thinking what is the problem in that area 0:20:42.320 --> 0:20:46.720 I'm solving with AI? And if AI is even appropriate? 0:20:47.280 --> 0:20:49.280 Because we want to make sure that you use AI 0:20:49.440 --> 0:20:52.320 not just because it's a cool, hot toy in the market, 0:20:52.359 --> 0:20:56.640 but you are convinced that it can significantly enhance the 0:20:56.760 --> 0:21:00.600 user experience of your customers in that area. And once 0:21:00.640 --> 0:21:03.160 you have an answer to those all these four questions, 0:21:03.240 --> 0:21:06.640 then maybe you have a good candidates to start applying AIT. 0:21:07.560 --> 0:21:11.320 And what about from the side of project managers who 0:21:11.320 --> 0:21:14.359 are trying to just keep up with how fast things 0:21:14.400 --> 0:21:18.720 are changing, how fast innovation is happening, Like, what advice 0:21:18.720 --> 0:21:19.919 would you give those people? 0:21:20.520 --> 0:21:24.800 My advice would be focus on agility. This is a 0:21:24.840 --> 0:21:28.520 market that is evolving rapidly and the winners of the 0:21:28.600 --> 0:21:32.080 market would be those that are able to take advantage 0:21:32.080 --> 0:21:35.320 of the best the market can offer at any point 0:21:35.320 --> 0:21:38.320 of time. So in order to do that, they need 0:21:38.359 --> 0:21:46.639 to be open to experimentation, continuous learning, and to rapidly 0:21:46.960 --> 0:21:48.520 adopting the new ideas. 0:21:49.720 --> 0:21:53.159 And when you think about the future and GENAI, is 0:21:53.240 --> 0:21:57.120 there a particular, say problem that you are most excited 0:21:57.160 --> 0:21:57.679 to solve. 0:21:58.359 --> 0:22:01.160 I think that would be productive. If you look into 0:22:01.280 --> 0:22:04.680 the stats that are out there, there are surveys that 0:22:04.960 --> 0:22:08.639 confirm that sixty to seventy percents of the time of 0:22:08.640 --> 0:22:14.639 our employees can be potentially enhanced to the productivity gains 0:22:14.640 --> 0:22:18.600 of generative For example, I personally myself use my product 0:22:18.640 --> 0:22:21.800 for content generation a lot, so the time that it 0:22:21.920 --> 0:22:27.080 frees up can be potentially put into generating a higher 0:22:27.200 --> 0:22:31.199 value work. And because of that, I'm super excited with 0:22:31.680 --> 0:22:36.240 all the opportunities that it represents for enterprises to go 0:22:36.320 --> 0:22:40.280 and dedicate the time of the employees to higher value items. 0:22:40.520 --> 0:22:45.280 Great. Okay, a couple Granite specific questions. So what are 0:22:45.359 --> 0:22:49.280 like the key things you want the world to know about. 0:22:48.960 --> 0:22:55.720 Granted Granite is open, trusted, and targeted. Two ways to 0:22:55.800 --> 0:22:59.760 think about openness. One open as open weights it's a 0:22:59.840 --> 0:23:02.960 very lebe for public to download. And the second one 0:23:03.320 --> 0:23:08.720 is open as in there is less restrictions on how 0:23:08.800 --> 0:23:12.120 the customers can legally use these models for a range 0:23:12.160 --> 0:23:15.440 of use cases. We have released Grantite open source models 0:23:15.440 --> 0:23:19.280 on their Apache license that is enabling a large range 0:23:19.320 --> 0:23:23.240 of use cases. The second one was trusted. We talked 0:23:23.240 --> 0:23:27.480 about that like it's rooted in the trustworthy governance process 0:23:27.520 --> 0:23:30.880 that we established thereund how we are training these models 0:23:31.280 --> 0:23:34.199 and the responsibility that we take for these models. And 0:23:34.280 --> 0:23:38.080 the third one is targeted, targeted for enterprise. We talked 0:23:38.119 --> 0:23:42.320 about like exposing Granted to enterprise data or the domain 0:23:42.440 --> 0:23:46.159 specific Granted some of them like Cobalt Java Translation that 0:23:46.280 --> 0:23:51.320 is targeting to solve specific enterprise needs and that's granite, 0:23:51.359 --> 0:23:53.200 so open, trusted and targeted. 0:23:53.920 --> 0:23:55.720 So there are a lot of models out in the 0:23:55.760 --> 0:23:58.880 world all of a sudden, right, it's a crowded market. 0:23:59.440 --> 0:24:02.320 Where does it fit in that universe? What is the 0:24:02.359 --> 0:24:03.240 market for granted? 0:24:04.240 --> 0:24:08.120 We talked about the enterprise market shifting away from very 0:24:08.240 --> 0:24:13.199 large general purpose models to target a smaller models, and 0:24:13.320 --> 0:24:18.040 Granted is a small model that enterprise can pick up 0:24:18.320 --> 0:24:23.240 and customize on their proprietary data to create something that 0:24:23.359 --> 0:24:27.359 is differentiated for a target use case. So Granted is 0:24:27.400 --> 0:24:32.120 well suited as a small, domain specific business, ready tailored 0:24:32.160 --> 0:24:37.360 for business and trained on enterprise data to solve enterprise questions. 0:24:37.840 --> 0:24:40.720 You mentioned small as one of the things that granted 0:24:40.840 --> 0:24:45.880 is why is that useful in some contexts for enterprise 0:24:45.960 --> 0:24:46.959 for businesses. 0:24:47.800 --> 0:24:52.280 The larger the model, the larger computer resources it requires. 0:24:52.960 --> 0:24:58.199 It translates to increased latency that's your response time. It 0:24:58.240 --> 0:25:04.840 translates to increase costs, and in translates to increase carbon 0:25:04.880 --> 0:25:09.399 footprint and energy consumption. So at the scale of enterprise transactions. 0:25:09.440 --> 0:25:11.959 When you move to production and you want to scale, 0:25:12.640 --> 0:25:17.800 some of these challenges can be multiple times stronger, like 0:25:17.920 --> 0:25:21.200 costs can add up, the energy consumption can be a 0:25:21.280 --> 0:25:24.879 serious thing, and the latency is depending on the application, 0:25:25.560 --> 0:25:31.840 can be a showstopper and blockier because for longer, larger models, 0:25:31.840 --> 0:25:35.359 more powerful models, it just takes the way longer time 0:25:35.560 --> 0:25:37.440 to process and calculate the output. 0:25:37.520 --> 0:25:41.359 For you, we are going to finish up with a 0:25:41.400 --> 0:25:45.879 speed round and I want you to just answer with 0:25:45.960 --> 0:25:48.359 the first thing that comes to mind. Don't overthink this, Okay, 0:25:48.640 --> 0:25:54.440 complete this sentence. In five years, AI will be invisible. Ah, 0:25:54.480 --> 0:25:56.480 I like that. What do you mean by that? 0:25:56.960 --> 0:26:01.520 Today? AI is everywhere? But if you ask my kids 0:26:01.560 --> 0:26:05.240 at home, they know AI. But if you say very like, 0:26:05.280 --> 0:26:07.680 how do you use AI, they don't know the answer 0:26:07.800 --> 0:26:12.040