WEBVTT - When two become 2degrees - managing a major systems merge 0:00:00.120 --> 0:00:03.480 It's been a big year from mergers and acquisitions. As 0:00:03.520 --> 0:00:07.280 a tough economy, competitors to join forces and try and 0:00:07.320 --> 0:00:10.119 achieve scale here and beyond Australasia. 0:00:10.200 --> 0:00:12.520 But what does it mean in practical terms when two 0:00:12.520 --> 0:00:16.280 companies merge? How do you spice everything together in an 0:00:16.400 --> 0:00:17.000 orderly way? 0:00:17.200 --> 0:00:19.959 Well? Two degrees is in the process of doing just 0:00:20.280 --> 0:00:23.800 that following its merger with Vocus to create New Zealand's 0:00:23.840 --> 0:00:25.840 third largest telecoms operator. 0:00:25.960 --> 0:00:28.920 On the sponsored episode of the Business at Tech, Stephen Kerjier, 0:00:29.160 --> 0:00:32.240 two degrees is chief information officer, talks to us about 0:00:32.280 --> 0:00:35.960 the major job of bringing two large businesses together with 0:00:36.159 --> 0:00:39.600 two million users on the platform and what he's learned 0:00:39.640 --> 0:00:42.600 so far in the two Degrees Vocus merger. 0:00:42.360 --> 0:00:45.159 And Stephen joins us. Now, Steven, thanks so much for 0:00:45.240 --> 0:00:47.600 coming on the Business of Tech. How are you doing. 0:00:47.880 --> 0:00:49.640 I'm doing great. Thanks for having me. Yeah, gat to 0:00:49.640 --> 0:00:50.159 be here soon. 0:00:50.200 --> 0:00:52.280 We're going to get into some really interesting things that 0:00:52.320 --> 0:00:57.160 you've been working on around integration of these telecommunications businesses 0:00:57.200 --> 0:01:00.320 from Vocus to two degrees. So lots of interesting stuff 0:01:00.360 --> 0:01:03.240 to cover there. But you're also joining us for our 0:01:03.480 --> 0:01:06.360 usual sort of run around the big tech headlines off 0:01:06.360 --> 0:01:09.400 the week and a Talco story that's a particular interest 0:01:09.480 --> 0:01:13.399 to you. The Commerce Commission is asking our big telcos, 0:01:13.440 --> 0:01:17.640 including two Degrees, One End, Zed and Spark, to start 0:01:17.680 --> 0:01:21.200 showing coverage maps. Now I thought they already did this. 0:01:21.280 --> 0:01:23.560 You go onto your website, you'll see a coverage map there, 0:01:23.600 --> 0:01:26.800 but they're looking for something a bit more extensive. What's 0:01:26.880 --> 0:01:27.440 your take on this. 0:01:28.000 --> 0:01:30.920 Yeah, I think we've all done it, but in different ways. 0:01:31.400 --> 0:01:33.959 So we have a coverage map on our website. I 0:01:33.959 --> 0:01:37.080 think it's actually a really good thing. It provides consistency 0:01:37.120 --> 0:01:40.280 to our consumers on coverage, so when they sign up, 0:01:40.319 --> 0:01:42.720 they know what they're actually getting. I think for two 0:01:42.720 --> 0:01:45.000 Degrees in particular, how we've always been the challenger in 0:01:45.040 --> 0:01:48.280 the market. One of our bigger issues is network perception. 0:01:48.520 --> 0:01:50.080 You know, do you actually have the coverage or not? 0:01:50.160 --> 0:01:53.240 Given our history, and we've invested in our network significantly 0:01:53.520 --> 0:01:56.200 over the last three to five years and we have 0:01:56.320 --> 0:01:58.400 parity now, so we actually think it's a great thing 0:01:58.440 --> 0:02:01.440 for us to show. So, you know, is coverage poor, 0:02:01.480 --> 0:02:04.120 is it good? Is it great? That transparency I think 0:02:04.200 --> 0:02:07.320 is really important for consumers. How hard is to do 0:02:07.600 --> 0:02:08.520 is probably another question. 0:02:09.160 --> 0:02:11.720 That's the thing, because the maps are there, the physical 0:02:12.080 --> 0:02:14.280 you know, your engineers go out and map this is 0:02:14.280 --> 0:02:17.680 where the coverage is. But then putting a rating on 0:02:17.720 --> 0:02:20.440 that is this is really good quality coverage, This is 0:02:20.480 --> 0:02:22.160 a bit miginal. I guess that's where it will be 0:02:22.160 --> 0:02:23.120 a little bit contentious. 0:02:23.160 --> 0:02:25.200 And how do you actually measure it because I'm sure 0:02:25.200 --> 0:02:27.720 there would be fluctuations and it would depend on capacity 0:02:27.760 --> 0:02:30.560 of the towers and things like that. So how are 0:02:30.560 --> 0:02:32.280 you thinking about that nuance? 0:02:32.919 --> 0:02:36.160 Yeah, in one way, you've got kind of network kind 0:02:36.160 --> 0:02:39.120 of measures, you know, signal strength, and those can be 0:02:39.240 --> 0:02:42.280 quite well defined. But I think ultimately its customer experience 0:02:42.360 --> 0:02:44.680 is the most important part of it. So I think 0:02:44.720 --> 0:02:48.480 coverage maps will give you a broad view, but it 0:02:48.560 --> 0:02:51.800 may not show kind of end to end network experience. 0:02:51.880 --> 0:02:54.639 So I think I'll give us enough of a good 0:02:54.680 --> 0:02:57.320 measure for consumers to make the right decisions, particularly in 0:02:57.360 --> 0:03:00.760 the more rural areas. But our focus atally around customer 0:03:00.840 --> 0:03:03.120 experience and how can we measure that kind of that 0:03:03.200 --> 0:03:06.079 true end to end view, which is around collecting the 0:03:06.160 --> 0:03:10.880 right data from the handset, either crowdsourced or you can 0:03:10.880 --> 0:03:13.960 also get some really interesting telemetry from the network as 0:03:14.000 --> 0:03:17.360 well on the user experience and also independent testing of 0:03:17.400 --> 0:03:21.840 the network. So recently we've actually done really well in 0:03:21.880 --> 0:03:24.800 some independent tests, so that's another measure. But from the 0:03:24.800 --> 0:03:27.919 Commerce Commission, it's really just get a baseline on what 0:03:28.320 --> 0:03:30.880 good coverage is and make it consistent, which I think 0:03:30.919 --> 0:03:32.520 is actually a great thing to do, but it is 0:03:33.080 --> 0:03:34.880 probably only one measure. It's not going to be the 0:03:34.920 --> 0:03:37.640 whole story to your question. Yeah. 0:03:37.720 --> 0:03:39.640 Yeah, And the other part of it is that the 0:03:39.680 --> 0:03:43.040 commissioner is looking to have an exit right put into 0:03:43.040 --> 0:03:46.000 contract so that if the customer signs up and they 0:03:46.000 --> 0:03:48.120 find that, like I think it was thirty percent of 0:03:48.160 --> 0:03:51.480 the small businesses that they surveyed that the coverage they're 0:03:51.480 --> 0:03:53.800 not happy with what they're getting, that there is a 0:03:53.840 --> 0:03:57.600 clause saying I want to exit based on this issue. Now, 0:03:57.600 --> 0:04:00.240 two degrees is one of the tolcoes that's already that 0:04:00.280 --> 0:04:03.600 in their contract for some time. Do you find that 0:04:03.720 --> 0:04:06.120 is something that's triggered a lot, or is it just 0:04:06.400 --> 0:04:09.520 gives you the confidence to bring on for customers to 0:04:09.640 --> 0:04:13.000 join you, Like, how does that work out in the field. 0:04:13.480 --> 0:04:17.159 Yeah, well, I say bring it on. Yeah, we've already 0:04:17.160 --> 0:04:21.160 had it. You know, sort of money back guarantee network reliability. 0:04:21.279 --> 0:04:24.479 More recently we've been doing more advertising on that, like 0:04:24.480 --> 0:04:27.159 a network blind test. You can actually try the network out, 0:04:27.320 --> 0:04:30.760 give it a go again. We're really focused on our 0:04:30.839 --> 0:04:35.000 network's fantastic. Now it's parody across all the players, but 0:04:35.080 --> 0:04:37.839 we have this network perception issue so that when customers 0:04:37.839 --> 0:04:39.520 get on our network they actually love it. 0:04:39.600 --> 0:04:42.239 So yeah, that was a big thing, getting to network 0:04:42.240 --> 0:04:46.480 parody as a third player, considering how long it took 0:04:46.560 --> 0:04:50.160 for both Spark and Vodafone to really flesh out coverage. 0:04:50.200 --> 0:04:52.760 There was a lot of complaints about those networks for 0:04:53.160 --> 0:04:55.440 a long time to even in semi rural places, even 0:04:55.600 --> 0:04:59.320 urban areas was patchy coverage. So I guess in some ways, 0:04:59.320 --> 0:05:00.960 coming from behind and as a third player, it was 0:05:00.960 --> 0:05:02.520 a bit easier because you knew some of the pain 0:05:02.560 --> 0:05:05.600 points they'd been through. But still a huge undertaking. 0:05:05.480 --> 0:05:09.839 Significant undertaking, and it keeps going right, it never stops. 0:05:10.040 --> 0:05:13.280 We're investing hundreds of millions dollars in our network every year. 0:05:13.960 --> 0:05:17.520 We've got a five G modernization program that we're well 0:05:17.560 --> 0:05:19.840 in flight. We started late for five G compared to 0:05:19.880 --> 0:05:22.039 our competitors, but we're well on track. I think we 0:05:22.080 --> 0:05:23.920 may be one of the ones to finish first, just 0:05:23.960 --> 0:05:26.760 in terms of our approach. So yeah, it never stops 0:05:26.920 --> 0:05:29.560 and it's never perfect, Like network coverage is always a 0:05:30.080 --> 0:05:33.560 thing where you're always looking to improve. But yeah, coming 0:05:33.600 --> 0:05:36.920 from the origins of basically starting from nothing, right, two 0:05:36.960 --> 0:05:39.919 degrees is a challenger, effectively a startup. We're a really 0:05:39.920 --> 0:05:43.839 big startup now and we still think like a startup, 0:05:44.560 --> 0:05:47.679 but quite an incredible journey from what was a position 0:05:47.760 --> 0:05:50.040 of not owning a network to where we are now, 0:05:50.080 --> 0:05:54.120 where we own over two thousand cell towers, We've got 0:05:54.160 --> 0:05:57.560 fiber assets that are four thousand, six hundred kilometers. The 0:05:57.600 --> 0:05:59.680 network's out of the equation for us, so our focus 0:05:59.720 --> 0:06:02.440 is now on customer experience and software and all that 0:06:02.480 --> 0:06:06.000 great stuff. It takes hard work and a commitment to 0:06:06.040 --> 0:06:08.960 just keep investing for a long term strategy. 0:06:09.200 --> 0:06:11.880 And of course you've got the text satellite stuff comes 0:06:12.040 --> 0:06:13.920 as well, so you've got the coverage map, but then 0:06:14.120 --> 0:06:16.280 you're going to have where the big gray spots are 0:06:16.320 --> 0:06:18.960 at the moment, the ability to send text messages initially 0:06:19.000 --> 0:06:20.960 anyway via satellite. 0:06:20.680 --> 0:06:23.080 That's right, Yeah, and that's the exciting part coming up, 0:06:23.320 --> 0:06:25.680 which we've partnered with link as I think you know 0:06:26.760 --> 0:06:30.359 and still trialing that. And the time for that to 0:06:30.360 --> 0:06:32.039 be effective is just how many birds you can have 0:06:32.080 --> 0:06:33.919 in the sky, how many satellites you can have in 0:06:33.960 --> 0:06:36.039 the sky. So yeah, we're really excited about that to 0:06:36.120 --> 0:06:40.520 really plug those other areas of coverage. And I'll also 0:06:40.560 --> 0:06:42.760 say it's an industry effort as well. You've got the 0:06:42.839 --> 0:06:46.240 Rural Connectivity Group that has been looking at how we 0:06:46.279 --> 0:06:49.200 do things more effective, so it's not just three players 0:06:49.200 --> 0:06:54.640 going out it themselves. We actually do collaborate significantly and 0:06:54.720 --> 0:06:57.600 that's going really well as well. RCG went over five 0:06:57.680 --> 0:07:01.560 hundred sites in terms of the real connectivity areas, so 0:07:01.920 --> 0:07:04.200 that's been really good. And complement that with a low 0:07:04.240 --> 0:07:08.159 Earth orbit for the celt satellite. So I think you 0:07:08.279 --> 0:07:11.200 have a really great story about more ubiquitous coverage in 0:07:11.240 --> 0:07:11.800 New Zealand. 0:07:11.880 --> 0:07:15.680 Yeah, you said you were thinking you'll possibly be completed 0:07:15.680 --> 0:07:18.160 your five G roll out ahead maybe some of the others. 0:07:18.160 --> 0:07:20.600 What does that actually look like. Would it be would 0:07:20.720 --> 0:07:24.880 have equitable coverage to for G or is it separate 0:07:25.200 --> 0:07:27.160 like more specified area rollout. 0:07:27.600 --> 0:07:31.240 Yeah, it's a great question. I think everyone starts with 0:07:31.360 --> 0:07:33.840 the higher population areas, right, that's where you start with 0:07:34.320 --> 0:07:37.440 terms of just needing more capacity, but there's also being 0:07:37.520 --> 0:07:40.320 a program of work to get five G in rural towns, 0:07:40.360 --> 0:07:43.560 working again with the government, so that's been really successful. 0:07:43.560 --> 0:07:47.600 Again that's part of investment strategy to get better coverage 0:07:47.600 --> 0:07:51.840 into rural towns. But yeah, our approach has been basically 0:07:51.840 --> 0:07:55.080 to modernize everything on that cell tart once. So not 0:07:55.160 --> 0:07:57.920 deploying just five G but modernizing four G and five 0:07:58.000 --> 0:08:02.600 G simultaneously means to actually get better four G experience 0:08:02.640 --> 0:08:06.480 and a five G experience at the same time, which 0:08:06.520 --> 0:08:08.320 I think is a really good strategy for us. It 0:08:08.360 --> 0:08:10.920 does mean probably it's a little bit slower initially because 0:08:10.920 --> 0:08:12.640 you're having to do a bit more work, but in 0:08:12.640 --> 0:08:15.520 the long run it's a great strategy for us to 0:08:15.560 --> 0:08:16.200 move forward on. 0:08:16.880 --> 0:08:18.640 So we'll keep an eye on those coverage maps to 0:08:18.680 --> 0:08:19.960 ComCom wants them within a year. 0:08:20.240 --> 0:08:24.040 Yes, yeah, the team are busy already working through it timeframes. 0:08:24.120 --> 0:08:26.440 Yeah, so we'll keep an eye on out. Another big 0:08:26.480 --> 0:08:29.280 story making headlines this week, which is sort of relevance 0:08:29.320 --> 0:08:33.800 to us, the Australian government has proposed mandatory guide rails 0:08:33.840 --> 0:08:39.520 for use of artificial intelligence. They want require testing, transparency 0:08:39.600 --> 0:08:43.000 about how AI is being used, and accountability. There's a 0:08:43.040 --> 0:08:47.040 ten point checklist that they have floated. This is out 0:08:47.080 --> 0:08:50.160 for public consultation for the next month or so. We 0:08:50.200 --> 0:08:53.360 haven't moved to this step yet, and I think the 0:08:53.400 --> 0:08:56.920 government will look quite closely at this. NBA has laid 0:08:56.920 --> 0:09:00.720 out sort of the roadmap that it's following, and Collins 0:09:00.880 --> 0:09:04.880 has said that she would like a risk based, proportional 0:09:04.920 --> 0:09:09.520 and light touch regulatory regime around AI. Here they'll only 0:09:09.559 --> 0:09:12.080 create AI legislation if there's a real need to, So 0:09:12.120 --> 0:09:14.720 there's a lot of parity sort of going on with 0:09:14.880 --> 0:09:18.920 the Australians. What's your take having looked at this already 0:09:19.000 --> 0:09:22.560 using AI at two degrees and the practicalities of trying 0:09:22.559 --> 0:09:23.960 to conform to something like this. 0:09:24.360 --> 0:09:26.240 I think what I've seen was really doing is actually 0:09:26.320 --> 0:09:28.640 really good because it is a risk based approach and 0:09:28.679 --> 0:09:33.640 actually focused on the higher risk areas, but also holistically, 0:09:33.720 --> 0:09:36.320 they're looking to invest in AI and actually accelerate it, 0:09:36.320 --> 0:09:39.840 particularly in government, So I think that's a really great thing. Yeah, 0:09:39.840 --> 0:09:42.440 for us, we've already been doing a lot in this space. 0:09:42.520 --> 0:09:45.960 So when we compare our AI policy and responsible use 0:09:46.160 --> 0:09:49.480 and all the great things around governance for AI. It 0:09:49.600 --> 0:09:51.440 very much aligns to what we're already doing at two 0:09:51.480 --> 0:09:57.200 degrees trust and transparency, having a human loop. Yeah, the 0:09:57.360 --> 0:10:00.719 explainability aspect, which is hard for a black box. But 0:10:00.720 --> 0:10:04.360 I think it's also important to recognize AI of the 0:10:04.440 --> 0:10:07.800 past is very different to AI now with large language models. 0:10:07.800 --> 0:10:11.720 So regulation previously was effectively based on like machine learning, 0:10:12.200 --> 0:10:16.680 are you applying decisions to HR process or to a 0:10:16.679 --> 0:10:21.520 criminal activity or churn predictions and how are you using it? 0:10:21.559 --> 0:10:24.280 But large language model is actually very different to that, 0:10:24.600 --> 0:10:28.720 although I wouldn't use them for actually making autonomous decisions yet, 0:10:28.720 --> 0:10:32.320 but decision support is really great. So I think a 0:10:32.320 --> 0:10:34.600 lot of the regulation is effectively the AI of the 0:10:34.640 --> 0:10:37.600 past that I've actually seen trying to control that. But 0:10:37.640 --> 0:10:40.280 the AI of the future is very different, And what 0:10:40.360 --> 0:10:43.600 the Australian proposal is is actually taking more of a 0:10:43.600 --> 0:10:46.240 holistic view of that, kind of breaking down what is 0:10:46.240 --> 0:10:49.199 a developer of a model, you're training a pre trained model, 0:10:49.240 --> 0:10:50.960 and what you need to do because that's a higher 0:10:51.040 --> 0:10:53.000 risk area. You know, if you don't have the guardrails on, 0:10:53.080 --> 0:10:56.880 you could create an unmitigated AI l em that could 0:10:56.880 --> 0:11:01.280 do anything versus someone who's deploying LM into an organization, 0:11:01.320 --> 0:11:04.760 which is effectively where we're at. And then also just 0:11:04.880 --> 0:11:07.240 users of AI. You know, we're all using it every day. 0:11:07.280 --> 0:11:10.440 I think nearly everyone is, So I think what they've 0:11:10.440 --> 0:11:13.120 done is actually broken it down quite pragmatically into those 0:11:13.120 --> 0:11:16.719 different areas and focusing on the higher risk components. So yeah, 0:11:16.720 --> 0:11:18.800 I think it's actually really good. It aligns with what 0:11:18.920 --> 0:11:21.920 we're doing. I don't like more audits, so I think 0:11:21.920 --> 0:11:24.240 there was a mention in the proposal around actually auditing 0:11:25.559 --> 0:11:29.200 entities around it. So having a more integrated approach where 0:11:29.360 --> 0:11:32.120 it's actually part of existing audits and compliance, I think 0:11:32.200 --> 0:11:33.560 is a better way to go. And I think they 0:11:33.600 --> 0:11:36.040 actually proposed a few options have it more integrated as 0:11:36.080 --> 0:11:37.240 well versus separate. 0:11:37.920 --> 0:11:40.680 It seems like when you look at AI risk, it 0:11:40.720 --> 0:11:43.840 doesn't immediately jump out that Talco would particularly be one 0:11:43.840 --> 0:11:46.200 of the more risky areas we are using AI. But 0:11:46.280 --> 0:11:50.200 then when you add that large language model factor into 0:11:50.240 --> 0:11:54.320 it for things like customer service or whatever else you 0:11:54.400 --> 0:11:57.280 might use for decision making support, that does kind of 0:11:57.760 --> 0:12:01.200 enhance the risk a step up by default of using 0:12:01.480 --> 0:12:04.480 those models. So how are you thinking about that? At 0:12:04.520 --> 0:12:09.240 two degrees in terms of maintaining your level of I 0:12:09.240 --> 0:12:12.000 guess soft compliance because we don't have that regulation, but 0:12:12.320 --> 0:12:15.400 compliance with your own internal guidelines. 0:12:16.360 --> 0:12:20.080 So we did deploy straight away quite early on, obviously 0:12:20.120 --> 0:12:22.360 the explosion of chat gipt and everyone got really excited 0:12:22.360 --> 0:12:26.880 about it, an AI responsible use policy with the intent 0:12:27.000 --> 0:12:29.920 to actually embrace the technology, not to kind of stop 0:12:29.960 --> 0:12:33.360 people using it, but put some guardrails around it so 0:12:33.400 --> 0:12:36.520 people can use it safely. So that was kind of 0:12:36.520 --> 0:12:42.160 immediately done and create more awareness around use of AI. 0:12:42.520 --> 0:12:44.760 So a lot of our work has been just creating, 0:12:45.320 --> 0:12:50.240 just doing lunch and learns and awareness training. So we 0:12:50.320 --> 0:12:52.160 do want to invite people to use it for their 0:12:52.240 --> 0:12:53.800 daily work. I think that's the best way to test 0:12:53.840 --> 0:12:56.520 and learn get ready for what I think is actually 0:12:56.520 --> 0:12:59.680 a huge you know, we're in a new revolution of technology. 0:13:00.240 --> 0:13:02.439 We're just we're trying to get ready and taking many 0:13:02.559 --> 0:13:05.280 different stakeholders on that journey, your board who are probably 0:13:05.400 --> 0:13:09.120 very risk adverse, you know, the use of AI just 0:13:09.160 --> 0:13:11.960 in simple things like if you're putting Microsoft Copilot on 0:13:12.000 --> 0:13:14.640 the teams meeting and the people that you invite into 0:13:14.679 --> 0:13:18.520 it will automatically get all the summarization notes and you 0:13:18.559 --> 0:13:21.520 may not want them all to so just being aware of, 0:13:21.760 --> 0:13:23.480 you know, the technology and what it could do in 0:13:23.559 --> 0:13:26.120 terms of transparency of information, we may not want it, 0:13:26.760 --> 0:13:30.840 and I think ethically we've just our main purpose is 0:13:30.880 --> 0:13:34.760 to put the customer first. So when you think about 0:13:34.760 --> 0:13:37.199 your customer experience as your north star, that's your guiding 0:13:37.280 --> 0:13:40.200 light for anything. It doesn't necessarily have to be AI, right, 0:13:40.200 --> 0:13:43.280 it could be any technology. When you put the customer first, 0:13:43.280 --> 0:13:45.840 that's the most important thing for us. So that's kind 0:13:45.880 --> 0:13:50.000 of our main principle under the use of AIS. You know, 0:13:50.120 --> 0:13:52.320 is it going to actually make a better experience for 0:13:52.360 --> 0:13:53.000 our customers. 0:13:54.280 --> 0:13:56.760 You've got to foster that trust, ye You've it's got 0:13:56.760 --> 0:13:59.080 to be a high trust system and part of that, 0:13:59.120 --> 0:14:01.400 you know, this is an interesting point that they've floated 0:14:01.720 --> 0:14:05.920 across the Tasman be transparent with other organizations across the 0:14:06.080 --> 0:14:09.240 entire AI supply chain. So being able to explain to 0:14:09.280 --> 0:14:13.400 your customers, Okay, we might be using something of AWS 0:14:13.559 --> 0:14:16.760 or Azure, which is underpinned by open AI. We need 0:14:16.800 --> 0:14:18.160 to be able to tell you a little bit and 0:14:18.240 --> 0:14:23.080 have confidence that our suppliers are giving uugh enough information 0:14:23.600 --> 0:14:25.800 to be confident that we're not going to breach privacy 0:14:26.400 --> 0:14:29.320 or that. So I guess that'll be something that increasingly 0:14:29.320 --> 0:14:32.400 you'll be looking at through the entire supply chain. What 0:14:32.440 --> 0:14:34.640 AI are we using and do we know enough about it? 0:14:35.160 --> 0:14:38.560 Yeah, definitely, And how using our data? We use a 0:14:38.560 --> 0:14:41.960 lot of third party systems that probably hasn't changed in 0:14:42.040 --> 0:14:45.600 risk profile. You've got to effectively trust. If you're using 0:14:45.600 --> 0:14:48.400 a cloud provider like Microsoft or Amazon, you have to 0:14:48.440 --> 0:14:53.400 trust what they're saying around their data use even something 0:14:53.480 --> 0:14:57.440 like open ai GPT four, which may be a misconceptionist 0:14:57.480 --> 0:14:59.800 because I think people might think of AI or as 0:14:59.840 --> 0:15:02.480 a entity. So when you put data in there, it's 0:15:02.520 --> 0:15:05.960 just learning everything, but it's still just a computer program 0:15:05.960 --> 0:15:09.160 in some respects, or a data program. And there are 0:15:09.240 --> 0:15:12.120 features in there where you can specifically say don't train 0:15:12.200 --> 0:15:14.880 on my data or use it privacy features like you 0:15:14.880 --> 0:15:18.760 would get with anything like Dropbox or anything. How using 0:15:18.960 --> 0:15:21.840 your email Gmail, for example, you may not trust Google 0:15:21.880 --> 0:15:23.720 to be training on your data either, but there are 0:15:23.760 --> 0:15:25.960 ways to actually opt in or out. So I think 0:15:26.040 --> 0:15:27.920 that's you do have an element of trust in the 0:15:27.920 --> 0:15:31.400 supply chain definitely, which we've already had I think in 0:15:31.440 --> 0:15:36.520 the past. So whether organizations trust these big tech companies 0:15:36.520 --> 0:15:39.680 to do that is another question. You've got themes around 0:15:39.720 --> 0:15:43.720 more sovereign AI and private AI as definitely building momentum 0:15:43.760 --> 0:15:46.800 as well. You're seeing that globally, seeing what France is 0:15:46.800 --> 0:15:49.160 doing that EU are doing, So I could see a 0:15:49.200 --> 0:15:52.560 future actually where we do have more sovereign AI use, 0:15:53.360 --> 0:15:55.560 which I think is actually not a bad thing, because 0:15:55.600 --> 0:15:57.640 the future of AIS, I think will be actually a 0:15:57.760 --> 0:16:01.480 zoo of them. There'll be LMS for various things. You've 0:16:01.480 --> 0:16:04.800 got your obviously your big foundational frontier models, but you've 0:16:04.840 --> 0:16:07.000 obviously got a huge amount of open source models. So 0:16:07.240 --> 0:16:09.080 I think it's going to be more of an ecosystem 0:16:09.360 --> 0:16:12.880 and more of a hybrid approach going forward, which I 0:16:12.880 --> 0:16:14.920 think we have to be ready for and regulation needs 0:16:14.960 --> 0:16:18.360 to make sure that is safe. 0:16:17.720 --> 0:16:22.120 And small language models for very specific purposes, and of 0:16:22.160 --> 0:16:25.000 course we've got the hyperscale is setting up here, so 0:16:25.280 --> 0:16:28.960 the option of keeping all your data within New Zealand's 0:16:29.440 --> 0:16:32.000 borders will be attractive as well from a privacy point 0:16:32.000 --> 0:16:32.320 of view. 0:16:32.520 --> 0:16:36.320 Absolutely, yeah. Are we using Amazon bedrock for a lot 0:16:36.360 --> 0:16:39.000 of our kind of experimentation and what we're doing today 0:16:39.720 --> 0:16:44.600 which has the LMS based in Australia, but you also 0:16:44.600 --> 0:16:47.320 don't get the latest models that often either. That's also 0:16:47.360 --> 0:16:52.040 a challenge with these entities. They do deploy them in 0:16:52.080 --> 0:16:54.440 their main regions first and they come down to our region. 0:16:55.760 --> 0:16:57.280 So we still want to be on the bleeding edge 0:16:57.280 --> 0:17:00.520 as well and experimenting. So yeah, we're also are minting 0:17:00.640 --> 0:17:04.040 with open source models, creating kind of those foundations to 0:17:04.080 --> 0:17:07.879 be able to have optionality. Earlier in the year, we 0:17:07.920 --> 0:17:11.879 released a chatbot, which was a generative AI chatbot to 0:17:12.000 --> 0:17:14.879 our customers in a test and learn approach. We actually 0:17:14.880 --> 0:17:17.080 deployed it into Slingshot, which is one of our brands 0:17:17.160 --> 0:17:20.560 still rather than our bigger two Degree space, just to 0:17:20.600 --> 0:17:23.359 really understand it's actually quite a dangerous thing to do. 0:17:23.440 --> 0:17:26.400 In some respects it can hallucinate. We put a lot 0:17:26.400 --> 0:17:29.280 of effort in testing guardrails and making sure it was 0:17:29.320 --> 0:17:32.160 defined in its own context so it couldn't go out 0:17:32.160 --> 0:17:35.200 of the context. My own team we're testing it. You know, 0:17:35.880 --> 0:17:38.480 you know, what would Steve buy from two Degrees or 0:17:38.520 --> 0:17:40.840 from Slingshot? Would you like a pair of shoes with 0:17:40.920 --> 0:17:44.439 that or something? And we had a lot of learnings 0:17:44.440 --> 0:17:47.960 from that, so we've built kind of an orchestration guardrail 0:17:48.960 --> 0:17:52.399 ecosystem around it to ensure that the chatbot doesn't stray 0:17:52.440 --> 0:17:54.960 off too far, which is a great learning for us, 0:17:55.160 --> 0:17:59.160 and it also taught us many other things like our 0:17:59.200 --> 0:18:02.960 knowledge bases need improving data is a huge aspect to 0:18:03.040 --> 0:18:07.200 get value out of these things. So it's accelerated our 0:18:07.280 --> 0:18:10.480 view on data modernization and knowledge management along the way. 0:18:10.520 --> 0:18:13.800 So it's I think every organization should go on this 0:18:13.960 --> 0:18:17.200 journey and actually test learn scale, just try it out, 0:18:17.320 --> 0:18:19.159 take a few risks, but make sure you've got the 0:18:19.200 --> 0:18:22.240 guardrails in place. That one is an example where it's 0:18:22.280 --> 0:18:25.239 actually customer facing. I think a lot of organizations are 0:18:25.240 --> 0:18:27.920 doing internal based chatbots as well, which is what we're doing, 0:18:28.320 --> 0:18:29.960 which is more is a safe way to do it 0:18:29.960 --> 0:18:30.359 as well. 0:18:30.560 --> 0:18:33.120 Did you have any moments of like real shock when 0:18:33.119 --> 0:18:35.960 a customer tried to do something or found a way 0:18:35.960 --> 0:18:37.879 to break the bot in any kind of way that 0:18:37.960 --> 0:18:40.719 you suddenly you had to deal with. 0:18:40.920 --> 0:18:43.879 I think we got most of the guardrails pretty well 0:18:44.160 --> 0:18:48.560 sorted what it did highlights because we do have the 0:18:48.640 --> 0:18:51.600 human and the loop still and looking at the actual 0:18:51.640 --> 0:18:55.080 responses and actually measuring how effective it is. It just 0:18:55.119 --> 0:18:58.040 really highlighted our knowledge management wasn't as good as it 0:18:58.080 --> 0:19:01.400 should be. So our knowledge base is our help desk articles, 0:19:01.440 --> 0:19:04.680 which is what the model is using to refine its answers. 0:19:05.400 --> 0:19:08.320 So when we had some results that we thought weren't 0:19:08.400 --> 0:19:11.879 quite right, it's actually we're already saying that to our customers. 0:19:12.119 --> 0:19:16.200 So it's actually helped our feedback loop. So not necessarily 0:19:16.280 --> 0:19:19.760 anything too wrong. Yeah, I think there are some cases 0:19:19.800 --> 0:19:21.919 where people tried to say, you know what I like 0:19:21.960 --> 0:19:23.919 some would you like frise with that or you know 0:19:24.280 --> 0:19:26.760 a few extra things, and the bot did actually go 0:19:26.880 --> 0:19:29.919 off and try to help. It really wants to please people, 0:19:30.119 --> 0:19:33.760 you know, that's the thing. So we had a few 0:19:33.760 --> 0:19:36.840 instances where it did stray off a little bit to 0:19:36.920 --> 0:19:39.600 try and please our customers because that was you know, 0:19:39.640 --> 0:19:43.120 it was kind of told to do that. So yeah, 0:19:43.160 --> 0:19:45.359 it's never perfect. I think. I think these things do 0:19:45.480 --> 0:19:48.600 hallucinate a little bit, so that's why it's effect. You'd 0:19:48.600 --> 0:19:50.400 still need the human and the loop, which is part 0:19:50.440 --> 0:19:55.399 of the Australian proposal and human oversight. Do you need 0:19:55.440 --> 0:19:58.680 to pull the trigger as well. We've got a within 0:19:58.720 --> 0:20:01.680 our own policy like a something does terribly go wrong 0:20:01.720 --> 0:20:03.879 with it we can just turn it off immediately. So 0:20:03.960 --> 0:20:06.840 we've got, yeah, the big red button, the control loop. 0:20:07.560 --> 0:20:11.160 Yeah, So that's what the Aussies will decide on. They've 0:20:11.160 --> 0:20:14.199 got three options in terms of how they do this 0:20:14.280 --> 0:20:16.280 from a framework point of view. They might have an 0:20:16.320 --> 0:20:18.919 AI act like the EU, or they might just tweak 0:20:19.480 --> 0:20:24.320 existing legislation. Does it sort of fill you with trepidation 0:20:24.440 --> 0:20:28.080 the idea of dedicated legislation being introduced. Would you prefer 0:20:28.160 --> 0:20:30.840 to see using existing legislation or is it much of 0:20:30.880 --> 0:20:31.639 a muchness to you? 0:20:32.400 --> 0:20:34.160 I don't think there's any right or wrong answer here 0:20:34.240 --> 0:20:38.639