WEBVTT - Smart Talks with IBM: How open source can democratize AI 0:00:00.160 --> 0:00:02.840 Hey everyone, it's Robert and Joe here. Today we've got 0:00:02.880 --> 0:00:04.760 something a little bit different to share with you. It 0:00:04.840 --> 0:00:08.039 is a new season of the Smart Talks with IBM 0:00:08.160 --> 0:00:09.160 podcast series. 0:00:09.640 --> 0:00:11.639 Today we are witnessed to one of those rare moments 0:00:11.680 --> 0:00:14.400 in history, the rise of an innovative technology with the 0:00:14.400 --> 0:00:18.680 potential to radically transform business and society forever. The technology, 0:00:18.760 --> 0:00:22.240 of course, is artificial intelligence, and it's the central focus 0:00:22.239 --> 0:00:24.840 for this new season of Smart Talks with IBM. 0:00:25.320 --> 0:00:28.400 Join hosts from your favorite Pushkin podcasts as they talk 0:00:28.480 --> 0:00:31.680 with industry experts and leaders to explore how businesses can 0:00:31.720 --> 0:00:35.400 integrate AI into their workflows and help drive real change 0:00:35.400 --> 0:00:38.240 in this new era of AI. And of course, host 0:00:38.280 --> 0:00:40.479 Malcolm Gladwell will be there to guide you through the 0:00:40.479 --> 0:00:42.640 season and throw in his two cents as well. 0:00:43.120 --> 0:00:46.200 Look out for new episodes of Smart Talks with IBM 0:00:46.400 --> 0:00:49.559 every other week on the iHeartRadio app, Apple Podcasts, or 0:00:49.600 --> 0:00:53.360 wherever you get your podcasts. And learn more at IBM 0:00:53.479 --> 0:01:03.360 dot com, slash smart Talks, Pushkin. 0:01:08.959 --> 0:01:12.600 Hello, Hello, welcome to Smart Talks with IBM, a podcast 0:01:12.600 --> 0:01:17.840 from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Gladwell. This 0:01:17.920 --> 0:01:21.240 season We're diving back into the world of artificial intelligence, 0:01:21.520 --> 0:01:24.920 but with a focus on the powerful concept of open 0:01:25.280 --> 0:01:30.440 its possibilities, implications, and misconceptions. We'll look at openness from 0:01:30.520 --> 0:01:33.520 a variety of angles and explore how the concept is 0:01:33.600 --> 0:01:37.200 already reshaping industries, ways of doing business, and a very 0:01:37.280 --> 0:01:41.039 notion of what's possible. In today's episode, I sat down 0:01:41.040 --> 0:01:44.880 with mo Duffy, software engineering manager at red Hat, who 0:01:44.920 --> 0:01:48.600 works on instruct Lab, a project co developed by red 0:01:48.600 --> 0:01:52.360 Hat and IBM. Most shared with me how this new 0:01:52.440 --> 0:01:57.680 initiative is revolutionizing AI training, making it not only more accessible, 0:01:57.880 --> 0:02:02.200 but also more inclusive. In this project, unique in the industry, 0:02:02.400 --> 0:02:07.760 allows developers to submit incremental contributions to one base AI model, 0:02:08.040 --> 0:02:12.240 creating a continuous loop of development, much like normal open 0:02:12.280 --> 0:02:17.639 source software. By leveraging community contributions and IBM's cutting edge 0:02:17.680 --> 0:02:20.840 granite models, Mo in the team of ibmrs and red 0:02:20.840 --> 0:02:24.359 hatters are paving the way for a future where AI 0:02:24.480 --> 0:02:28.880 development is a communal endeavor. Our insights into open source 0:02:28.919 --> 0:02:33.959 software extend beyond technical proficiency to the profound impact of 0:02:34.120 --> 0:02:37.720 collaborative effort. At the heart of Moe's work is a 0:02:37.720 --> 0:02:42.519 belief in democratizing technology, ensuring that AI becomes a tool 0:02:42.919 --> 0:02:47.320 accessible to all. So let's explore how MOE, red Hat 0:02:47.360 --> 0:02:51.880 and IBM are empowering individuals and businesses alike to reshape 0:02:51.880 --> 0:03:01.600 the future of technology through collaboration and innovation. MO, thank 0:03:01.639 --> 0:03:03.800 you for joining me today. Thank you so much, for 0:03:04.040 --> 0:03:08.480 I have just about the most Irish name ever. I 0:03:08.560 --> 0:03:12.040 do very proudure you weren't born in Ireland. 0:03:12.240 --> 0:03:13.239 No, my grandparents. 0:03:13.320 --> 0:03:15.200 Oh your grandparents, So I see, where did you grow up? 0:03:15.600 --> 0:03:16.639 New York Queens? 0:03:16.960 --> 0:03:20.200 Oh you're la see. So tell me a little bit 0:03:20.200 --> 0:03:22.960 about how how you got to red hat. What was 0:03:22.960 --> 0:03:23.480 your path? 0:03:23.960 --> 0:03:26.480 When I was in high school, it was a chatty girl, 0:03:26.800 --> 0:03:29.280 teenage girl on the phone. We had one phone line. 0:03:29.639 --> 0:03:32.440 My older brother was studying at the local state college 0:03:32.440 --> 0:03:34.720 computer science, and he had to tell that end to 0:03:35.040 --> 0:03:37.920 compile his homework. One phone line, and I'm on it 0:03:37.960 --> 0:03:40.920 all the time. He got very frustrated and he needed 0:03:40.920 --> 0:03:43.480 a compiler to do his homework. So he bought red 0:03:43.480 --> 0:03:47.920 Hat Linux from a CompUSA, brought it home and that 0:03:48.040 --> 0:03:49.960 was on the family computer. So I learned Linux and 0:03:50.000 --> 0:03:52.720 I started playing around with it. I really liked it 0:03:52.760 --> 0:03:56.680 because you could customize everything, like the entire user interface. 0:03:57.040 --> 0:03:59.200 You could actually modify the code of the programs you 0:03:59.240 --> 0:04:01.480 were using to do what you wanted. And for me, 0:04:01.560 --> 0:04:03.760 it was really cool because especially when you're a kid 0:04:03.800 --> 0:04:05.560 and like people tell you this is the way things 0:04:05.560 --> 0:04:07.320 are and you just have to deal with it, it's 0:04:07.400 --> 0:04:08.920 nice to be like, I'm going to make things the 0:04:09.000 --> 0:04:12.280 way I want, modify the code and playing. Yeah, it 0:04:12.360 --> 0:04:14.200 was amazing and it was just such a time and 0:04:14.240 --> 0:04:17.240 like before it was cool, I was doing it and 0:04:18.040 --> 0:04:20.000 what I saw on that is sort of the potential 0:04:20.040 --> 0:04:23.080 like number one of like a community of people working together. 0:04:23.080 --> 0:04:26.160 And like the Internet existed, it was slow, it involved modems, 0:04:26.560 --> 0:04:28.480 but there were people that you could talk to who 0:04:28.520 --> 0:04:32.279 would give you tips and you'd share information, and this 0:04:32.400 --> 0:04:35.320 collaborative building something together is really something special. 0:04:35.440 --> 0:04:35.640 Right. 0:04:36.000 --> 0:04:39.640 I could file a complaint to whatever large software company 0:04:39.680 --> 0:04:42.080 made whatever software I was into, or I could go 0:04:42.080 --> 0:04:44.279 to an open source software community and be like, hey, guys, 0:04:44.320 --> 0:04:46.719 I think we should do this. I'm like, yeah, okay, 0:04:46.720 --> 0:04:48.880 I'll help. I'll pitch in. So you don't feel powerless, 0:04:48.880 --> 0:04:50.240 you feel like you can have an impact, and that 0:04:50.320 --> 0:04:54.200 was really exciting to me. However, open source software has 0:04:54.360 --> 0:04:57.599 a reputation for not having the best user interface, not 0:04:57.640 --> 0:05:02.159 the best user experience. So I ended up studying Computer 0:05:02.200 --> 0:05:05.480 science and Electronic media dual major, and then I did 0:05:05.560 --> 0:05:09.440 human computeraction as my master's And my thought was, wouldn't 0:05:09.440 --> 0:05:13.120 it be nice if this free software accessible to anybody, 0:05:13.440 --> 0:05:15.440 if it was easier to use, some more people could 0:05:15.520 --> 0:05:18.160 use it and take advantage of it. And so, long 0:05:18.200 --> 0:05:21.479 story short, I ended up going to Red Hat saying, Hey, 0:05:21.480 --> 0:05:22.880 I want to learn how you guys work. Let me 0:05:22.920 --> 0:05:25.719 embed in your team draft out of my graduate program. 0:05:25.760 --> 0:05:27.520 And I'm like, I want to do this for a living. 0:05:27.640 --> 0:05:30.240 This is cooler. So I thought this is the way 0:05:30.279 --> 0:05:31.920 to go, and I've been there ever since. They haven't 0:05:31.920 --> 0:05:32.760 been able to get rid of me. 0:05:34.839 --> 0:05:37.560 To backtrack just a little bit, you were talking about 0:05:37.600 --> 0:05:41.280 the sense of community that surrounds this way of thinking 0:05:41.320 --> 0:05:45.039 about software. Talk a little bit more about what that 0:05:45.080 --> 0:05:47.840 community is like, the benefits of that community, why it 0:05:47.920 --> 0:05:48.720 appeals to you. 0:05:49.200 --> 0:05:51.520 Sure, well, you know part of the reason I actually 0:05:51.640 --> 0:05:54.720 ended up going to the graduate school track. Suddenly you're 0:05:54.720 --> 0:05:57.920 a peer of your professors and you're working side by 0:05:58.000 --> 0:06:01.040 side with them. At some point they retire and you're 0:06:01.040 --> 0:06:04.599 in the next generation. So it's sharing information, building on 0:06:04.640 --> 0:06:07.320 the work of others in sort of this cycle that 0:06:07.520 --> 0:06:11.880 extends past human lifespan and in the same way, like 0:06:11.960 --> 0:06:15.359 the open source model is very similar, but you're actually 0:06:15.440 --> 0:06:18.080 you're building something, and it's something in me. I'm just 0:06:18.440 --> 0:06:20.840 really attracted, Like I don't like talking about stuff. I 0:06:21.000 --> 0:06:24.960 like doing stuff with open source software. The software doesn't 0:06:24.960 --> 0:06:28.599 cost anything, the code is out there, generally uses open 0:06:28.640 --> 0:06:32.039 standards for the file formats. I can open up files 0:06:32.080 --> 0:06:34.680 that I created and open source tools as a high 0:06:34.720 --> 0:06:38.279 school student today because they were using open formats and 0:06:38.320 --> 0:06:40.800 that software still exists. I can still compile the code 0:06:40.880 --> 0:06:44.080 and it's an active community project. Like these things can 0:06:44.120 --> 0:06:46.800 outlast any single company in the same way that the 0:06:46.839 --> 0:06:49.400 academic community has been going on for so many years 0:06:49.440 --> 0:06:51.960 and hopefully we'll continue moving on. So it's sort of 0:06:52.040 --> 0:06:54.919 like not just the community around it, but just the 0:06:55.000 --> 0:06:58.119 knowledge sharing and also bringing up the next generation as well. 0:06:58.160 --> 0:06:59.680 Like all of that stuff really appealed to me. And 0:06:59.760 --> 0:07:02.440 also so at the center of it the fact that 0:07:02.880 --> 0:07:05.960 we could democratize it by following this open source process 0:07:06.320 --> 0:07:08.320 and feel like we have some control. We're not at 0:07:08.360 --> 0:07:11.400 the mercy of some faceless corporation making changes and we 0:07:11.440 --> 0:07:13.520 have no impact. Like that really appealed to me too. 0:07:14.320 --> 0:07:19.400 For those of us who are not software phisionados, take 0:07:19.400 --> 0:07:23.800 a step backwards and give me a kind of description 0:07:23.920 --> 0:07:26.800 of terms. What's the opposite of open source proprietary? 0:07:26.880 --> 0:07:28.360 Proprietary is what we say. 0:07:28.280 --> 0:07:32.679 So specifically and practically, the difference would be what between 0:07:32.720 --> 0:07:34.840 something that was opens us in something that was proprietary. 0:07:34.920 --> 0:07:37.520 Sure, so there's a lot of difference. So with open 0:07:37.560 --> 0:07:41.520 source software you get these rights when you're given the software, 0:07:41.600 --> 0:07:44.000 you get the right to be able to share it. 0:07:44.080 --> 0:07:46.520 And depending on the lot, different licenses that are considered 0:07:46.520 --> 0:07:49.520 open source have different little things that you have to 0:07:49.520 --> 0:07:55.320 be aware of. With proprietary code, it's one copyright the company. 0:07:55.360 --> 0:07:57.720 Even a lot of times, when you sign your employment 0:07:57.720 --> 0:08:00.160 contract for a software company and you write code for them, 0:08:00.360 --> 0:08:02.000 you don't own it. You sign over your rights to 0:08:02.040 --> 0:08:03.960 the company. So if you leave the company, the code 0:08:03.960 --> 0:08:06.280 doesn't go with you. It stays in the ownership of 0:08:06.280 --> 0:08:08.920 that company. So then one like one company buys out 0:08:08.960 --> 0:08:11.080 another and kills a product, that code's gone. 0:08:11.240 --> 0:08:15.160 It's gone for a business, Why would a business want 0:08:15.240 --> 0:08:19.040 to be have open source code as opposed to a proprietary. 0:08:18.400 --> 0:08:20.480 Well for the same reasons. Like, say you're a business. 0:08:20.960 --> 0:08:24.840 You've invested all this money into this software platform, right 0:08:25.320 --> 0:08:28.440 and you've upskilled your employees on it, and it's a 0:08:28.480 --> 0:08:31.320 core part of your business, and then a few years 0:08:31.360 --> 0:08:34.679 later that company goes out of business or something happens, 0:08:34.840 --> 0:08:38.199 or even something less drastic. You really need this future, 0:08:38.520 --> 0:08:41.640 But for the company that makes the software, it's not 0:08:41.640 --> 0:08:44.480 in their best interests. It's not worth the investment. They're 0:08:44.520 --> 0:08:46.440 not going to do it. How do you get that future? 0:08:46.800 --> 0:08:49.320 You either have to completely migrate to another solution, and 0:08:49.360 --> 0:08:51.280 this is something it's core at your business, that's going 0:08:51.320 --> 0:08:53.439 to be a big deal to migrate. But if it's 0:08:53.480 --> 0:08:57.240 open source, you could either hire a team of experts. 0:08:57.440 --> 0:08:59.880 You could hire software engineers who are able to go 0:09:00.120 --> 0:09:03.319 do this for you. Go in the upstream software community, 0:09:03.800 --> 0:09:06.800 implement the feature that you want, and it'll be rolled 0:09:06.840 --> 0:09:09.400 into the next version of that company software. So even 0:09:09.440 --> 0:09:12.559 if that company didn't want to implement the feature, if 0:09:12.600 --> 0:09:15.640 they did it open source, they would inherit that feature 0:09:15.679 --> 0:09:18.160 from the upstream community, is what we call it. So 0:09:18.240 --> 0:09:20.880 you have some control over the situation. If it's open source, 0:09:20.960 --> 0:09:25.120 you have an opportunity to actually affect change in the product, 0:09:25.400 --> 0:09:27.400 and you could then pick it up or pay somebody 0:09:27.400 --> 0:09:29.520 else to pick it up, or another company could form 0:09:29.559 --> 0:09:32.160 and pick it up and keep it going. So there's 0:09:32.160 --> 0:09:34.640 more possibilities. If it's open source, it's more like it's 0:09:34.679 --> 0:09:36.240 like an insurance policy almost. 0:09:36.280 --> 0:09:39.840 So innovation from the standpoint of the customer, innovation is 0:09:39.880 --> 0:09:42.880 a lot easier when you're working in an open source environment. 0:09:43.040 --> 0:09:43.679 Absolutely. 0:09:44.080 --> 0:09:47.840 Yeah. So now at RedHat you're working with something called 0:09:47.840 --> 0:09:50.560 instruct lab. Tell us a little bit about what that is. 0:09:51.040 --> 0:09:53.280 So the thing that really excites me about getting to 0:09:53.320 --> 0:09:55.680 work on this project is AI is sort of that 0:09:55.760 --> 0:09:58.920 has been this scary thing for me because it's one 0:09:58.920 --> 0:10:02.040 of those things like in order to be able to 0:10:02.120 --> 0:10:07.239 pre train a model, you have to have unobtainium GPS, 0:10:07.920 --> 0:10:12.200 you have to have rich resources, It takes months, it 0:10:12.240 --> 0:10:17.200 takes expertise. There's a small handful of companies that can 0:10:17.240 --> 0:10:21.240 build a model from pre train to something usable, and 0:10:21.480 --> 0:10:23.800 it kind of feels like those early days when I 0:10:23.920 --> 0:10:26.640 was kind of delving in software in the same way. 0:10:26.679 --> 0:10:30.000 I think if more people could contribute to AI models, 0:10:30.800 --> 0:10:34.400 then it wouldn't be just influenced by whichever company had 0:10:34.440 --> 0:10:37.679 the resources to build it. And there's been a lot 0:10:37.679 --> 0:10:41.040 of emphasis on pre training models, so taking massive terabytes 0:10:41.160 --> 0:10:45.120 data sets, throwing them through masses of GPS over months 0:10:45.120 --> 0:10:48.440 of time, spending hundreds of millions of dollars to build 0:10:48.480 --> 0:10:51.320 a base model. But when instruct lab does is say okay, 0:10:51.559 --> 0:10:54.440 you have a base model. We're going to fine tune in. 0:10:54.480 --> 0:10:57.920 On the other end, it takes less compute resources. The 0:10:57.960 --> 0:11:00.240 way we've built in struck lab, you can play around 0:11:00.280 --> 0:11:02.760 with the technology and learn it on it off the 0:11:02.760 --> 0:11:06.200 shelf laptop that you can actually buy. So in this 0:11:06.320 --> 0:11:10.040 way we're enabling a much broader set of people to 0:11:10.160 --> 0:11:12.839 play with AI, to contribute it, to modify it. And 0:11:12.880 --> 0:11:15.840 I'll tell you one story from red Hat Succi, who 0:11:15.920 --> 0:11:20.360 is our chief diversity officer, very interested in inclusive language 0:11:20.360 --> 0:11:23.280 and open source software, doesn't have any experience with AI. 0:11:24.040 --> 0:11:26.079 We have a community model that we have an upstream 0:11:26.080 --> 0:11:28.680 project around for people to contribute knowledge and skills to 0:11:28.720 --> 0:11:30.840 the model. She's like, I want to teach the model 0:11:31.160 --> 0:11:34.120 how to use inclusive language, like replace this word with 0:11:34.160 --> 0:11:35.959 this word or this word with this word. OHM Like, oh, 0:11:36.000 --> 0:11:38.880 that's so cool. So She paired up with Nicholas who 0:11:38.960 --> 0:11:41.760 is a technical guy at red Hat, and they built 0:11:41.880 --> 0:11:44.920 and submitted a skill to the model that you can 0:11:45.000 --> 0:11:47.160 just tell the model, can you please take this document 0:11:47.240 --> 0:11:49.679 and translate this language to more inclusive language and it 0:11:49.720 --> 0:11:52.120 will do it. And they submitted it to the community. 0:11:52.200 --> 0:11:53.920 They were so proud. It was like, that's the kind 0:11:53.960 --> 0:11:56.360 of thing that, like, you know, maybe a company would 0:11:56.400 --> 0:11:58.840 be incentivized to do that, but if you have some 0:11:58.960 --> 0:12:02.560 tooling that's open source and something that anybody could access, 0:12:02.720 --> 0:12:05.320 than those communities could actually get together and build that 0:12:05.400 --> 0:12:06.640 knowledge into AI models. 0:12:06.880 --> 0:12:11.400 Just so understand, what you guys have is the structure 0:12:11.480 --> 0:12:15.760 for an AI system, And in other cases, individual companies 0:12:15.920 --> 0:12:19.880 own and train their own AI systems. It takes enormous 0:12:19.880 --> 0:12:22.920 amount of resources. They hoover up all kinds of information, 0:12:23.480 --> 0:12:26.640 train it according to their own hidden set of rules, 0:12:26.720 --> 0:12:31.160 and then a customer might use that for some price. 0:12:31.520 --> 0:12:33.440 What you're saying is, in the same way that we 0:12:33.520 --> 0:12:37.360 democratize the writing of software before, let's democratize the training 0:12:37.400 --> 0:12:41.040 of an AI system. So anyone can contribute here and 0:12:41.480 --> 0:12:45.160 teach the model the things that they're interested in teaching 0:12:45.160 --> 0:12:48.079 the model. I'm guessing correct me. On the one hand, 0:12:48.559 --> 0:12:50.680 this model, at least in the beginning, is going to 0:12:50.679 --> 0:12:53.840 have a lot fewer resources available to it. But on 0:12:53.880 --> 0:12:55.640 the other hand, it's going to have a much more 0:12:56.000 --> 0:12:58.000 diverse set of inputs. 0:12:58.440 --> 0:13:01.520 That's right. And the other thing is that IBM, basically 0:13:01.600 --> 0:13:04.280 is part of this project, has something called the Granite 0:13:04.320 --> 0:13:07.480 Model family, and they've donated some granite models. So these 0:13:07.520 --> 0:13:10.200 are the ones that take the months and terabytes of 0:13:10.280 --> 0:13:13.440 data and all the GPUs to train. So IBM has 0:13:13.480 --> 0:13:16.760 created one of those, and they have listed out and 0:13:16.800 --> 0:13:18.959 linked to the data sets that they used, and they 0:13:19.000 --> 0:13:21.920 talk about the relative proportions they used when pre training, 0:13:22.280 --> 0:13:24.280 so it's not just the black box. You know where 0:13:24.320 --> 0:13:27.160 the data came from, which is a pretty open position 0:13:27.200 --> 0:13:29.760 to take. That is what we recommend as the base. 0:13:29.840 --> 0:13:32.440 So you use the instruct lab tuning. You take this 0:13:32.520 --> 0:13:35.480 base granite model that IBM has provided, and you use 0:13:35.480 --> 0:13:37.920 the instruct lab tooling that red Hat works on, and 0:13:37.960 --> 0:13:40.280 you use that to fine tune the model to make 0:13:40.320 --> 0:13:41.960 it whatever you want. 0:13:42.480 --> 0:13:45.400 I want to go back to the partnership between IBM 0:13:45.480 --> 0:13:49.480 and red Hat here with them providing the granite model 0:13:50.080 --> 0:13:53.320 to your instruct lab Is this the first time red 0:13:53.360 --> 0:13:56.760 hat and IBM have collaborated like this, I think it's. 0:13:56.600 --> 0:13:59.920 Something that's been going on. Like another a product within 0:14:00.080 --> 0:14:02.560 the red hat family would be open Shift AI, where 0:14:02.559 --> 0:14:06.199 they collaborate a lot with IBM Research team, Like BLM 0:14:06.280 --> 0:14:08.240 is one of the components of that product that there's 0:14:08.440 --> 0:14:13.160 a nice kind of exchange and collaboration between the two companies. 0:14:13.920 --> 0:14:16.559 How large is the potential community of people who might 0:14:16.640 --> 0:14:18.240 contribute to instruct lab. 0:14:19.160 --> 0:14:21.720 It could be thousands of people. I mean, we'll see. 0:14:21.760 --> 0:14:25.520 It's early days. This is early technology that was invented 0:14:25.520 --> 0:14:28.000 at IBM Research that they partnered with us at red 0:14:28.000 --> 0:14:30.840 Hat to kind of build the software around it. There's 0:14:30.880 --> 0:14:33.160 still more to go, Like right now, we have a 0:14:33.240 --> 0:14:35.240 team in the community that's actually trying to build a 0:14:35.280 --> 0:14:38.800 web interface to make it easier for anybody to contribute. 0:14:38.960 --> 0:14:40.480 So we have a lot of those sort of user 0:14:40.520 --> 0:14:43.960 experience for the contributor to the model stuff to work 0:14:44.000 --> 0:14:46.680 out that we're still actively building on. But like my 0:14:46.880 --> 0:14:49.520 vision for it even is I like going back to 0:14:49.520 --> 0:14:52.600 that academic model of learning from what others and building 0:14:52.680 --> 0:14:55.560 upon it over time. It would be very good for 0:14:55.720 --> 0:14:58.560 us to sort of go out and try to collaborate 0:14:58.720 --> 0:15:01.520 with academics of all, like, hey, you know, the model 0:15:01.520 --> 0:15:05.000 doesn't know about your field, would you like to put 0:15:05.040 --> 0:15:07.440 something into the model about your field so it knows 0:15:07.480 --> 0:15:10.520 about it, or even you know, talk to the model 0:15:10.760 --> 0:15:13.160 it got it wrong, let's correct it. Can we lean 0:15:13.200 --> 0:15:15.320 on your expertise to correct it and make sure it 0:15:15.320 --> 0:15:18.400 gets it right and sort of use that community model 0:15:18.440 --> 0:15:22.480 as a way for everybody to collaborate because before instruct lab, 0:15:23.240 --> 0:15:26.680 my understanding is if you wanted to take a model 0:15:26.720 --> 0:15:28.800 that's open source license and play with it, you could 0:15:28.840 --> 0:15:30.520 do that. You could take a model kind of off 0:15:30.520 --> 0:15:33.560 the shelf from Hugging Face and fine tune it yourself. 0:15:33.960 --> 0:15:35.520 But it's a bit of a dead end because you 0:15:35.560 --> 0:15:38.120 made your contributions, but there's no way for other people 0:15:38.600 --> 0:15:41.520 to collaborate with you. So the way that we've built 0:15:41.560 --> 0:15:45.640 this is based on how the technology works. Everybody can 0:15:45.680 --> 0:15:48.040 contribute to it. This is something that you can keep 0:15:48.080 --> 0:15:49.520 growing and growing and growing over time. 0:15:49.840 --> 0:15:53.400 Yeah. Yeah, what's the level of expertise necessary to be 0:15:53.440 --> 0:15:54.160 a contributor? 0:15:54.760 --> 0:15:56.640 You don't need to be a data scientist and you 0:15:56.680 --> 0:15:59.600 don't need to have exotic hardware. Honestly, if you don't 0:15:59.600 --> 0:16:02.520 even have laptop hardware that meets SUSPEC for doing instruct 0:16:02.560 --> 0:16:05.720 labs laptop version. You can submit it to the community 0:16:05.800 --> 0:16:08.320 and then we'll actually build it for you. We have 0:16:08.400 --> 0:16:10.800 bots and stuff that do that, and we're hoping over 0:16:10.840 --> 0:16:13.320 time to make that more accessible, first by having a 0:16:13.440 --> 0:16:16.040 user interface and then maybe later on having a web service. 0:16:16.360 --> 0:16:19.560 Yeah, so give me an example of how a business 0:16:19.680 --> 0:16:21.720 might make use of instruct lab. 0:16:22.280 --> 0:16:24.800 One of the things that businesses are doing with AI 0:16:24.920 --> 0:16:28.600 right now is using hosted API services. They're quite expensive, 0:16:28.880 --> 0:16:31.680 but they're finding value, but it's hard given the amount 0:16:31.720 --> 0:16:34.000 of money they're spending. And one of the things that's 0:16:34.000 --> 0:16:35.840 a little scary about it too, is like you have 0:16:36.200 --> 0:16:40.560 very sensitive internal documents and you have employees maybe not 0:16:40.760 --> 0:16:43.640 understanding what they're actually doing because you know, how would 0:16:43.680 --> 0:16:47.080 you if you're not technical enough when you're asking said 0:16:47.720 --> 0:16:53.440 public web service AI model information about your copy pasting 0:16:53.560 --> 0:16:57.600 internal company documents. It's going across the Internet into another 0:16:57.640 --> 0:17:00.800 company's hands, and that company probably shouldn't have access to that. 0:17:01.280 --> 0:17:04.280 So what both RedHat and IBM in the space are 0:17:04.280 --> 0:17:07.640 looking at, like the instruct lab model is very modest. 0:17:07.640 --> 0:17:11.160 It's seven billion parameter model very small. It's very cheap 0:17:11.200 --> 0:17:14.600 to serve inference on a seven billion parameter model. It's 0:17:14.600 --> 0:17:18.120 competing with trillion parameter models that are hosted. You take 0:17:18.160 --> 0:17:21.000 this small model that is cheap to run inference on, 0:17:21.640 --> 0:17:25.560 you train it with your own company's proprietary data inside 0:17:25.600 --> 0:17:28.159 the walls of your company, on your own hardware. You 0:17:28.200 --> 0:17:31.480 can do all sorts of actual data analysis on your 0:17:31.480 --> 0:17:34.080 most sensitive data and have the confidence that has not 0:17:34.160 --> 0:17:35.080 left the premises. 0:17:35.920 --> 0:17:38.960 In that use case, you're not actually training the model 0:17:39.000 --> 0:17:42.240 for everyone. You're just taking it and doing some private 0:17:42.280 --> 0:17:44.720 stuff on it exactly, which doesn't leave the building. But 0:17:44.800 --> 0:17:49.320 that's separate from an interaction where you're doing something that 0:17:50.000 --> 0:17:51.200 contributes overall. 0:17:51.600 --> 0:17:54.200 Right, And that's something maybe that I should be more 0:17:54.240 --> 0:17:56.480 clear about is there's sort of two tracks here, and 0:17:56.560 --> 0:17:59.840 this is very red hat classic. You have your upstream 0:18:00.040 --> 0:18:02.960 community track and you have your business product tract. So 0:18:03.040 --> 0:18:07.040 the upstream community track is just enabling anybody to contribute 0:18:07.080 --> 0:18:09.000 to a model in a collaborative way and play with it. 0:18:09.440 --> 0:18:13.280 The downstream product business oriented track is now take that 0:18:13.400 --> 0:18:18.000 tech that we've honed and developed in the open community 0:18:18.680 --> 0:18:21.160 and apply it to your business knowledge and skills. 0:18:22.200 --> 0:18:26.040 This community driven approach marks a pivotal shift towards more 0:18:26.040 --> 0:18:31.840 accessible AI solutions. The contrast between externally hosted AI services 0:18:32.119 --> 0:18:35.320 and the open model enhanced by instruct lab underscores the 0:18:35.320 --> 0:18:40.440 potential for broader adoption of AI in diverse business contexts. 0:18:40.920 --> 0:18:44.320 She envisions a future in which technological innovation is more 0:18:44.359 --> 0:18:48.679 tailored to individual business needs, guided by principles of openness 0:18:48.800 --> 0:18:53.679 and security. To an imaginary case study, Sure, I'm a 0:18:53.760 --> 0:18:58.520 law firm, I'm an entertainment law I have one hundred 0:18:58.560 --> 0:19:02.879 clients who are big stars. They all have incredibly complicated contracts. 0:19:03.840 --> 0:19:08.280 I feed a thousand of my company's contracts from the 0:19:08.359 --> 0:19:12.200 last ten years into the model, and then every time 0:19:12.240 --> 0:19:14.800 I have a new contract, I ask the model, am 0:19:14.840 --> 0:19:17.520 I missing something? Can you go back and look through 0:19:17.560 --> 0:19:19.919 all our own contracts and show me a contract that 0:19:20.200 --> 0:19:23.439 is missing key components or exposes us to some liability. 0:19:24.320 --> 0:19:27.960 In that case, the model would know my law firm 0:19:28.200 --> 0:19:31.560 contracts really, really well. It's as if they've been working 0:19:32.080 --> 0:19:35.480 out my law firm. They're not distracted by other people's 0:19:35.520 --> 0:19:41.240 particular styles or a bunch of contracts from the utility industry, 0:19:41.400 --> 0:19:46.040 or the They know entertainment law contracts exactly. 0:19:46.160 --> 0:19:48.000 Yeah, and you can train it in your own image, 0:19:48.040 --> 0:19:51.800 your style of doing things. It's something that your company 0:19:51.880 --> 0:19:55.200 can produce that is uniquely helpful to you. No third 0:19:55.240 --> 0:19:57.720 party could do that because no third party understands how 0:19:57.720 --> 0:20:01.200 you do business and understands your his street in your documents. 0:20:01.520 --> 0:20:04.120 So it's sort of a way of getting value out 0:20:04.119 --> 0:20:06.000 of the stuff you already have sitting in a file 0:20:06.040 --> 0:20:08.200 cabinet somewhere. It's very cool. 0:20:08.480 --> 0:20:11.480 Yeah, give me a sort of a real world case 0:20:11.480 --> 0:20:14.720 study where you think the business use case would be 0:20:14.720 --> 0:20:19.320 really powerful. What's a business that really could see an 0:20:19.359 --> 0:20:23.360 advantage to using instruct lab in its way. 0:20:23.840 --> 0:20:26.119 The demo that I've given a couple of times at 0:20:26.119 --> 0:20:29.680 different events used an imaginary insurance company. So you say, 0:20:29.720 --> 0:20:33.639 you have this company, you have to recommend repairs for 0:20:33.720 --> 0:20:37.159 various types of claims. You've been doing this for years, 0:20:37.200 --> 0:20:40.040 you know. If you know the windshield's broken and you've 0:20:40.080 --> 0:20:42.920 gotten this type of accident and it's this model car, 0:20:43.119 --> 0:20:44.880 these are the kinds of things you want to look at. 0:20:45.560 --> 0:20:48.359 So you could talk to any insurance agent in the 0:20:48.359 --> 0:20:51.080 field and be like, oh, you know, it's a Tesla. 0:20:51.160 --> 0:20:53.280 You might want to look at the battery or something like. 0:20:53.400 --> 0:20:56.800 They'll have some latent knowledge just so you can take 0:20:56.840 --> 0:20:59.520 that and train it into a model. Honestly, I think 0:20:59.560 --> 0:21:02.840 these kind of new technologies are better when they're less visible. 0:21:03.440 --> 0:21:05.879 So say you have the claims agents in the field 0:21:05.960 --> 0:21:07.960 and they have this tool and they're kind of entering 0:21:07.960 --> 0:21:10.920