WEBVTT - Can bots improve our mental health? 0:00:00.009 --> 0:00:03.019 This podcast was made on the lands of the Wurundjeri people, 0:00:03.190 --> 0:00:06.349 the Woi-wurrung and the Bunurong. We'd like to pay respects 0:00:06.360 --> 0:00:09.270 to their elders, past and present and emerging. 0:00:14.399 --> 0:00:17.930 From the Melbourne School of Psychological Sciences at the University 0:00:17.940 --> 0:00:20.370 of Melbourne. This is PsychTalks. 0:00:22.620 --> 0:00:25.100 Welcome again to psych talks. And if you aren't familiar 0:00:25.110 --> 0:00:27.760 with us, in each episode we unpack the latest research 0:00:27.770 --> 0:00:30.709 in psychology and neuroscience with our incredible colleagues here at 0:00:30.719 --> 0:00:33.319 the University of Melbourne School of Psychological Sciences. 0:00:34.470 --> 0:00:37.400 To do this, I'm joined by my co-host associate Professor 0:00:37.409 --> 0:00:38.200 Cassie Hayward. 0:00:38.250 --> 0:00:41.880 Hi, Nick. Well, today we're welcoming back Professor Simon Dennis, 0:00:41.889 --> 0:00:44.950 who longer-time listeners might recognise from the very first series 0:00:44.959 --> 0:00:48.229 of PsychTalks. But this time we're exploring his latest research 0:00:48.240 --> 0:00:51.110 into how we might use AI - or artificial intelligence - 0:00:51.340 --> 0:00:53.639 to help us improve treatments for mental health. 0:00:53.650 --> 0:00:56.419 Sounds fantastic, Cassie. I can't wait to get stuck into it. 0:01:01.529 --> 0:01:03.220 Welcome back to PsychTalks, Simon. 0:01:03.459 --> 0:01:04.512 Uh, thanks for having me 0:01:04.512 --> 0:01:07.730 Simon, you now have about 30 years of experience working 0:01:07.739 --> 0:01:10.069 in AI, um, but let's start at the beginning of 0:01:10.080 --> 0:01:10.730 your journey. 0:01:11.110 --> 0:01:13.949 Yeah. So, um, so I should admit from the outset 0:01:13.959 --> 0:01:16.599 that my PhD was in computer science. I'm very sorry 0:01:16.610 --> 0:01:20.000 for those who are offended by that. It's been a long journey, 0:01:20.010 --> 0:01:22.545 so I've been in many psych departments around the world 0:01:22.555 --> 0:01:25.096 and found myself as a head of school of psychology 0:01:25.106 --> 0:01:28.365 in Newcastle for a while. But it's really just in 0:01:28.375 --> 0:01:32.195 the last year or so that I've started to think about, Um, 0:01:32.206 --> 0:01:36.486 how can I meld my understandings of AI and how, 0:01:36.496 --> 0:01:39.945 in particular large language models work with, um, kind of 0:01:39.956 --> 0:01:43.291 mental health and, uh, to utilise that in a way 0:01:43.302 --> 0:01:44.741 that's helpful for people. 0:01:44.751 --> 0:01:49.372 So, Simon, outside of your research within the academic space, 0:01:49.382 --> 0:01:53.872 I believe you're also building an application for wider use 0:01:54.561 --> 0:01:54.931 about that. Can you tell us about that? 0:01:55.022 --> 0:01:58.501 Yes, that's right. So we've incorporated a company called Mental Health Hub. 0:01:58.512 --> 0:02:01.891 And so this is a joint exercise with Caitlin Hitchcock 0:02:01.902 --> 0:02:06.139 and Michael Diamond, and what we're trying to do is 0:02:06.150 --> 0:02:09.740 to build, I guess the starts of a system for 0:02:09.750 --> 0:02:13.139 mental health support. Mental health support agents is how I'd 0:02:13.149 --> 0:02:16.538 like to say it. I think while I'm sure there's 0:02:16.550 --> 0:02:19.979 a large number of people who couldn't imagine going to 0:02:19.990 --> 0:02:23.490 replace a, uh, a human therapist with a bot. I 0:02:23.500 --> 0:02:26.929 think there's also a large group of people for whom 0:02:26.940 --> 0:02:29.669 talking to the bot is less intimidating than talking to 0:02:29.679 --> 0:02:33.008 another person. So, you know, it may well be that 0:02:33.020 --> 0:02:37.059 we're able to bring those people into the fold, I 0:02:37.070 --> 0:02:40.809 guess of mental health support in a way that wouldn't 0:02:40.820 --> 0:02:43.770 have otherwise been possible. So in the first instance, what 0:02:43.779 --> 0:02:47.080 we're doing is trying to think about things like cognitive 0:02:47.089 --> 0:02:52.130 behavioural therapy exercises. So trying to cut off the low 0:02:52.139 --> 0:02:54.800 hanging fruit, I guess, and think about, you know, how 0:02:54.809 --> 0:02:59.410 can we improve the rates with which people complete those exercises? 0:02:59.419 --> 0:03:03.990 Estimates vary, but some people think that it's around about 20% 0:03:04.000 --> 0:03:06.459 of people who actually complete the exercises that they're asked 0:03:06.470 --> 0:03:09.625 to do by their therapists. So I'm talking about, um, 0:03:09.633 --> 0:03:12.735 like pencil and paper exercises that a therapist might ask 0:03:13.163 --> 0:03:14.744 to do in between. 0:03:14.975 --> 0:03:16.264 But a human therapist? 0:03:16.274 --> 0:03:18.565 A human therapist would ask people to do them, but 0:03:18.574 --> 0:03:20.654 they wouldn't be there with them doing it. This would 0:03:20.663 --> 0:03:23.625 be the homework that they're doing in between sessions. Um, 0:03:23.964 --> 0:03:26.074 so yeah, so we were kind of thinking, OK, can 0:03:26.085 --> 0:03:29.664 we increase the rate at which people complete those exercises. 0:03:29.919 --> 0:03:32.220 But then what we're hoping to do, You know, we 0:03:32.229 --> 0:03:35.100 do have kind of pretty wide ambitions, and we would 0:03:35.110 --> 0:03:37.770 like to ramp it all the way up at some point. Um, 0:03:37.919 --> 0:03:40.000 you know, one of the things I would just like 0:03:40.009 --> 0:03:43.039 to emphasise is that we never kind of see the 0:03:43.050 --> 0:03:47.429 bot as being a replacement for therapists. What we're very 0:03:47.440 --> 0:03:50.160 focused on is how do we fit into the whole 0:03:50.169 --> 0:03:53.559 ecosystem here? We know that you know, the demand for 0:03:53.570 --> 0:03:58.300 therapy services way outstrips the supply. And so I think 0:03:58.376 --> 0:04:01.516 there's a lot of scope for us to kind of help. 0:04:01.526 --> 0:04:04.477 In the first instance, we're focused on the kind of 0:04:04.486 --> 0:04:08.026 mild to moderate depression and anxiety kinds of areas where 0:04:08.037 --> 0:04:11.897 the consequences are kind of not as severe as they 0:04:11.906 --> 0:04:15.477 might be in more complex cases. And also the challenge 0:04:15.486 --> 0:04:16.747 is not quite so severe. 0:04:16.837 --> 0:04:19.957 So I heard you used the expression CBT there Simon, 0:04:19.967 --> 0:04:21.986 can you just clarify what that stands for and what 0:04:21.997 --> 0:04:22.397 it is? 0:04:22.596 --> 0:04:26.356 Yeah. So it stands for cognitive behavioural therapy, and it's 0:04:26.486 --> 0:04:30.614 the style or approach to therapy I guess that we 0:04:30.623 --> 0:04:34.394 would say has the largest evidence base. So in terms 0:04:34.403 --> 0:04:37.653 of being efficacious, That's the one that most people point 0:04:37.664 --> 0:04:40.133 to and say, That's the kind of gold standard, I guess. 0:04:40.144 --> 0:04:44.364 Of course, there are many other approaches and other approaches 0:04:44.373 --> 0:04:47.164 have evidence bases as well. But that's the one that 0:04:47.174 --> 0:04:49.894 people would be most comfortable saying "Yes, absolutely." 0:04:49.903 --> 0:04:53.433 Do you know anything about what type of person would 0:04:53.444 --> 0:04:55.911 prefer talking to a bot? Is it an age thing? 0:04:55.920 --> 0:04:58.550 I know a lot of younger people, You know, would not 0:04:58.561 --> 0:05:00.640 want to talk to a real person but would very 0:05:00.651 --> 0:05:02.601 happily interact with the bot. Is it an age thing, 0:05:02.610 --> 0:05:04.110 or is it a personality thing? 0:05:04.351 --> 0:05:08.950 I suspect it's an age and gender thing. So, um so, yeah, definitely. 0:05:08.960 --> 0:05:12.590 The younger generations, they're just used to being on their phones. 0:05:12.601 --> 0:05:15.911 We actually provide our service through Facebook, so it's basically 0:05:15.920 --> 0:05:18.210 like talking to someone else on Facebook. So, I think 0:05:18.221 --> 0:05:20.270 that's part of it. You know, we see a big 0:05:20.281 --> 0:05:25.019 gender gap in terms of uptake of traditional therapy, and so, 0:05:25.029 --> 0:05:26.839 you know, we suspect that part of that is to 0:05:26.850 --> 0:05:29.380 do with the way that males feel about sharing with 0:05:29.390 --> 0:05:30.010 other people. 0:05:30.019 --> 0:05:32.299 So is that females are more happy to talk to 0:05:32.309 --> 0:05:33.859 a real person and males will...? 0:05:33.970 --> 0:05:37.299 Yeah, So females are much more likely, though, to engage 0:05:37.309 --> 0:05:38.769 with traditional therapy. Yeah. 0:05:38.880 --> 0:05:42.140 So for those of us who are still living in 0:05:42.149 --> 0:05:45.029 caves like myself, can you tell us more about the 0:05:45.040 --> 0:05:47.190 actual technology and how it works? And what is the 0:05:47.200 --> 0:05:51.630 nature of the delivery? Is it text? Is it voice? 0:05:51.678 --> 0:05:53.369 How's it gonna look? 0:05:53.380 --> 0:05:55.880 Yeah, OK, so there are many levels at which I 0:05:55.890 --> 0:06:00.059 could describe the technical, uh, the technical components. But let 0:06:00.070 --> 0:06:04.149 me just say that essentially what the computational model is 0:06:04.160 --> 0:06:07.649 doing is just doing prediction. So it's getting the words 0:06:07.660 --> 0:06:10.190 that have already been said. And it's just saying, What's 0:06:10.200 --> 0:06:12.079 the next best word to say? What's the next best 0:06:12.089 --> 0:06:15.178 word to say and just keeps, chaining them together that way. 0:06:15.190 --> 0:06:19.029 So the real advance that has occurred over the last 0:06:19.040 --> 0:06:22.289 five or six years has been in the way that 0:06:22.299 --> 0:06:25.910 we understand how to find the relevant information in the 0:06:25.920 --> 0:06:29.600 context that has appeared before. So previously, There was a 0:06:29.609 --> 0:06:31.950 lot of attention on just the very last thing that 0:06:31.959 --> 0:06:35.079 was said, but often it's the case that that it's 0:06:35.089 --> 0:06:38.500 information way back in the past that's relevant to making 0:06:38.510 --> 0:06:41.618 the current decision. So that's kind of the technical advance 0:06:41.630 --> 0:06:46.202 that has occurred. In terms of the way that it unfolds, 0:06:46.202 --> 0:06:49.808 so our bot is focused primarily on text. And so, 0:06:50.190 --> 0:06:52.399 as I said, we're operating through Facebook and so forth, 0:06:52.410 --> 0:06:54.329 so it really is literally a set of texts that 0:06:54.339 --> 0:06:57.730 come to you. But obviously there's all kinds of possibilities now, right? 0:06:57.738 --> 0:07:02.029 So we're seeing voice becoming very proficient. We're seeing a 0:07:02.040 --> 0:07:06.779 lot of stuff happening with image generation, also video and 0:07:06.790 --> 0:07:12.130 ongoing understanding of visual input. So, you know, I think 0:07:12.140 --> 0:07:14.260 the sky is the limit, really. 0:07:14.679 --> 0:07:19.670 One of the early criticisms of text-based analysis was that 0:07:19.850 --> 0:07:25.040 computers couldn't understand irony or sarcasm. Working in an Australian context. 0:07:25.049 --> 0:07:28.230 I imagine sarcasm is something that you need to understand 0:07:28.239 --> 0:07:32.519 from if a patient was talking to your bot and said, Yeah, 0:07:32.529 --> 0:07:35.239 I'm fine, but you could you, as a human could 0:07:35.250 --> 0:07:39.359 understand that that person was using that sarcastically. Does your model- 0:07:39.369 --> 0:07:42.989 Does your bot pick up on those sorts of language? 0:07:44.670 --> 0:07:48.350 Yeah, so I think there's two kind of answers to 0:07:48.359 --> 0:07:50.970 that question. The first one is that with these large 0:07:50.980 --> 0:07:55.929 language models, the understanding of irony and so forth that 0:07:55.940 --> 0:07:59.100 that you can get from the text, Um, that's not 0:07:59.109 --> 0:08:02.329 really an issue. So the, you know, they're perfectly capable 0:08:02.339 --> 0:08:05.089 of of doing that. They're picking the statistics out of 0:08:05.100 --> 0:08:08.000 the environment. And as long as those those, um, elements 0:08:08.010 --> 0:08:11.839 are in that statistical environment, they can capture it. The 0:08:11.850 --> 0:08:14.140 other issue the other issue that you were kind of 0:08:14.149 --> 0:08:17.799 alluding to, though, is that often that information is coming 0:08:17.809 --> 0:08:20.929 through intonation or something like that, not just the content 0:08:20.940 --> 0:08:24.290 of the text, right? And so that's where there's still 0:08:24.299 --> 0:08:26.630 quite a bit of work to be done because obviously, 0:08:26.640 --> 0:08:29.119 you've got to have the verbal stuff first before you 0:08:29.130 --> 0:08:32.479 can do that, a real therapist is using the nonverbals 0:08:32.489 --> 0:08:36.599 as well to make determinations about- and so obviously you're 0:08:36.609 --> 0:08:39.439 not going to achieve that through a text interface. So, 0:08:39.650 --> 0:08:42.098 you know, I think that that's a little bit further off. 0:08:42.109 --> 0:08:44.650 You know, I think realistically it's going to take us 0:08:44.659 --> 0:08:46.579 a while before we're going to be there. But, you know, 0:08:46.590 --> 0:08:50.080 we we're making progress. Progress has been much faster than 0:08:50.090 --> 0:08:51.750 anyone anticipated. I think so. 0:08:52.169 --> 0:08:56.589 So what would the advantages be as this develops further 0:08:56.599 --> 0:09:00.299 as it gets better at detecting all sorts of complexities? 0:09:00.309 --> 0:09:02.089 You know, interpersonal and emotional. 0:09:02.309 --> 0:09:05.750 So there are really quite a lot of reasons that 0:09:05.760 --> 0:09:08.500 you might want to do this. If you think about 0:09:08.510 --> 0:09:12.250 the demand versus supply, so way more demand than there 0:09:12.260 --> 0:09:15.469 is supply in terms of therapy services. To the extent 0:09:15.479 --> 0:09:18.500 that we can use these mechanisms to really kind of 0:09:18.510 --> 0:09:21.270 ramp up the total amount that's available to people, I 0:09:21.280 --> 0:09:26.330 think that's a really good thing. It's also about when because, unfortunately, 0:09:26.340 --> 0:09:29.679 people's mental health crises don't always happen between the hours 0:09:29.690 --> 0:09:33.330 of 9 to 5, and in fact, you know often 0:09:33.340 --> 0:09:35.880 they happen, um, in the early hours of the morning. 0:09:35.890 --> 0:09:40.010 And so the ability to respond exactly when the client 0:09:40.020 --> 0:09:41.590 needs you, I think is a really - is a 0:09:41.599 --> 0:09:44.890 really big advantage of these kinds of models. Hopefully, we 0:09:44.900 --> 0:09:46.819 can bring the costs way down as well. You know, 0:09:46.830 --> 0:09:50.049 accessibility is really a serious issue that I think we 0:09:50.059 --> 0:09:52.010 can address, and then there's like a whole bunch of 0:09:52.020 --> 0:09:56.835 kind of subsidiary advantages too. So, what we're hoping is 0:09:56.844 --> 0:09:59.965 we're going to get into the mental health process much 0:09:59.974 --> 0:10:02.494 earlier as a consequence of this. And so we can 0:10:02.505 --> 0:10:06.065 hopefully cut off problems before they become really intransient. 0:10:06.075 --> 0:10:09.275 And then the other thing is multilingual support. So you know, 0:10:09.284 --> 0:10:13.674 it's really quite difficult for non-English speakers in Australia to 0:10:13.684 --> 0:10:18.049 access therapy services in their own language. And it's often 0:10:18.059 --> 0:10:22.020 quite important because even people for whom their language comprehension 0:10:22.030 --> 0:10:25.650 is sufficient to kind of get around in everyday life, 0:10:25.659 --> 0:10:27.010 you know, so they can go to the shops and 0:10:27.020 --> 0:10:29.330 all that kind of stuff and kind of complete their 0:10:29.340 --> 0:10:32.750 activities of everyday living. When it comes to therapy conversations, 0:10:32.760 --> 0:10:37.890 it's often very nuanced, and they really struggle with those 0:10:37.900 --> 0:10:40.669 more nuanced conversations. And so, you know, if we can 0:10:40.679 --> 0:10:43.130 kind of meet them in their own language, I think 0:10:43.140 --> 0:10:45.868 that's you know, there's got to be advantages there. so 0:10:45.880 --> 0:10:48.760 lots of reasons why we're so like, super excited about 0:10:48.770 --> 0:10:49.738 trying to make it happen. 0:10:49.750 --> 0:10:52.429 The pricing thing is interesting as well. Isn't it? Because 0:10:52.440 --> 0:10:55.729 on one hand you could say I want to pay 0:10:55.739 --> 0:10:59.330 to have my problem solved, right, whether it's being solved 0:10:59.340 --> 0:11:02.119 by a human therapist or a bot, I'm paying for a 0:11:02.130 --> 0:11:05.669 resolution of my issue. But on the other hand, you 0:11:05.679 --> 0:11:09.179 can see that people would feel weird about paying as 0:11:09.190 --> 0:11:11.520 much for a bot therapist as they would for a 0:11:11.530 --> 0:11:14.179 human therapist. So where do you stand on the kind 0:11:14.190 --> 0:11:16.190 of pricing for bot therapy? 0:11:16.320 --> 0:11:18.760 Yeah, so it's interesting because, you know, one of the 0:11:18.770 --> 0:11:21.179 things we've been doing in the company is trying to 0:11:21.280 --> 0:11:23.549 query people about this and see, you know, what are 0:11:23.559 --> 0:11:26.760 their kind of kinds of attitudes? And, you know, when 0:11:26.770 --> 0:11:28.549 we first came in, we thought, you know, we had 0:11:28.679 --> 0:11:31.429 dollars in our eyes and we were thinking, oh, you know, well, 0:11:31.440 --> 0:11:34.729 if a if a therapy session is going to be, um, 0:11:34.900 --> 0:11:38.700 you know, $280 or $250 for an hour, how much 0:11:38.710 --> 0:11:41.750 can we charge? But what became apparent pretty quickly is 0:11:41.760 --> 0:11:43.890 that that's not the way that people think about it. 0:11:43.900 --> 0:11:46.770 So they actually think about the therapy more like kind 0:11:46.780 --> 0:11:49.974 of Spotify or Netflix or something like that. And I 0:11:49.984 --> 0:11:53.784 think that kind of embodies well, firstly that it's a bot, 0:11:53.875 --> 0:11:55.864 but also the different way that they're thinking that they 0:11:55.875 --> 0:11:58.454 would use it so it wouldn't be so much "I've 0:11:58.465 --> 0:12:01.265 got this deep problem that I'm going to, you know, 0:12:01.275 --> 0:12:03.794 work through in a detailed fashion the way I would 0:12:03.804 --> 0:12:06.755 with a human clinician." But more, you know, "I want 0:12:06.765 --> 0:12:09.194 to engage with a service that's going to kind of 0:12:09.205 --> 0:12:12.854 promote that mental well-being just in in general." I guess. 0:12:13.320 --> 0:12:16.020 And so it's a different kind of objective, I think. 0:12:16.229 --> 0:12:17.630 And so that's one of the things we're trying to 0:12:17.640 --> 0:12:19.679 come to terms with the company at the moment is 0:12:19.690 --> 0:12:21.140 what the people really want from us. 0:12:21.150 --> 0:12:24.020 So more like a subscription service that you just have. 0:12:24.030 --> 0:12:26.968 And then you can talk to your therapist bot kind 0:12:26.979 --> 0:12:30.099 of whenever you want, rather than thinking, OK, Thursday, four o'clock, 0:12:30.109 --> 0:12:32.444 I have my session? You can just pick it up 0:12:32.455 --> 0:12:33.094 at any time. 0:12:33.104 --> 0:12:36.885 That's right. Yeah, so we're trying to do cognitive behavioural therapy. 0:12:36.914 --> 0:12:40.554 How do you adapt that, you know, session-based approach that, 0:12:40.565 --> 0:12:44.174 as it's usually applied to this situation where people you 0:12:44.184 --> 0:12:47.113 know might do two or three interactions now and then 0:12:47.125 --> 0:12:49.294 later in the day, they do another couple of interactions 0:12:49.304 --> 0:12:51.705 and so forth, and, um, then they might go for 0:12:51.715 --> 0:12:53.885 a week without doing anything? And one of the things 0:12:53.895 --> 0:12:57.900 we've been focusing on, um, lately has been these kind 0:12:57.909 --> 0:13:02.210 of check ins. So how do we just spontaneously check 0:13:02.219 --> 0:13:04.650 in with people and understand? You know, what kind of 0:13:04.659 --> 0:13:06.949 questions should we be asking them and that kind of stuff? 0:13:06.960 --> 0:13:10.770 So having bot the prompt the person about how they're feeling, rather the other way around? 0:13:10.780 --> 0:13:12.789 Yeah, if I had to guess, I think that's going 0:13:12.799 --> 0:13:15.090 to end up being one of the key pieces of it, 0:13:15.099 --> 0:13:18.969 because obviously that's difficult for a clinician to do at scale. 0:13:18.979 --> 0:13:21.219 But we can do quite a lot of, and I 0:13:21.229 --> 0:13:23.030 suspect that's what people are really going to start to 0:13:23.039 --> 0:13:23.590 look for. 0:13:23.599 --> 0:13:26.804 And I think your point around getting in early before 0:13:26.815 --> 0:13:29.905 things kind of really spiral out of control is such 0:13:29.914 --> 0:13:33.275 a fascinating way of looking at the advantages of these 0:13:33.344 --> 0:13:36.424 bot therapy technologies and that if you can get in 0:13:36.434 --> 0:13:39.164 before and I don't know what the current waiting lists are, 0:13:39.174 --> 0:13:42.354 but it's a substantial amount of time when you can 0:13:42.364 --> 0:13:46.145 make a booking to get in to see a therapist. And, 0:13:46.155 --> 0:13:48.875 you know, in that time, obviously things might be spiralling 0:13:48.885 --> 0:13:51.385 out of control. So having a bot that can step 0:13:51.395 --> 0:13:55.489 in even if it's only kind of keeping things stable 0:13:55.500 --> 0:13:58.130 until if you need to kind of progress to see 0:13:58.140 --> 0:14:01.770 a therapist. But this idea of kind of just checking 0:14:01.780 --> 0:14:05.359 in every day or whenever you're feeling the need. I mean, 0:14:05.369 --> 0:14:08.010 that's just fascinating from a research perspective, too, to see, 0:14:08.020 --> 0:14:12.830 like these small doses of therapy versus one big dose 0:14:12.840 --> 0:14:13.929 over the week. 0:14:14.340 --> 0:14:18.900 Yeah, some people are somewhat sceptical about how effective therapy 0:14:18.909 --> 0:14:22.179 just is in general, like human therapy. And so you 0:14:22.190 --> 0:14:24.440 can you can think about well, why might that be 0:14:24.450 --> 0:14:26.599 the case? You know, we've been studying it for a while, 0:14:26.609 --> 0:14:28.729 you know, lots of lots of people are interested, and 0:14:28.739 --> 0:14:31.130 it may well be right that it's nothing to do with, 0:14:31.140 --> 0:14:33.919 you know, our understanding or anything like that. But it's 0:14:33.929 --> 0:14:36.640 more to do with just the logistics that if you 0:14:36.650 --> 0:14:39.820 can't get to people fast enough, um, it doesn't matter 0:14:39.830 --> 0:14:42.450 how good you are. Um, yeah. I mean, I don't 0:14:42.460 --> 0:14:44.409 think that's everything. All of it. But I think that 0:14:44.419 --> 0:14:46.039 could be an important part of it. 0:14:46.250 --> 0:14:48.880 This all raises the issue of, you know, what do 0:14:48.890 --> 0:14:52.200 actual clinicians think about this? Are they worried about competition? 0:14:52.210 --> 0:14:55.010 Do they think it's going to be a really good supplement? Um, 0:14:55.020 --> 0:14:58.570 if so, are they being naïve? Uh, tell us. 0:14:58.580 --> 0:15:01.429 So, Yeah. So we we've actually been doing a project 0:15:01.440 --> 0:15:05.960 where we've been going out to clinicians and collecting their opinions. Overall, 0:15:05.969 --> 0:15:11.500 the clinicians are basically right on the fence, so the 0:15:11.510 --> 0:15:15.070 mean is exactly in the middle of the scale. And 0:15:15.080 --> 0:15:16.789 I think a lot of that is driven by them 0:15:16.799 --> 0:15:19.780 not having a real good understanding of exactly how the 0:15:19.789 --> 0:15:22.080 technology works and so forth. So I think there's going 0:15:22.090 --> 0:15:25.239 to be a big education component to it. The concerns 0:15:25.250 --> 0:15:29.809 that they have are perfectly understandable. So privacy and security 0:15:29.820 --> 0:15:32.659 is a real key one. And when you look at 0:15:32.669 --> 0:15:36.280 the use cases, they are very interested in using it 0:15:36.289 --> 0:15:39.960 to produce case notes and using it to help them 0:15:39.969 --> 0:15:42.409 with their research of the literature to make sure that 0:15:42.419 --> 0:15:44.559 they're across what they need to be for this client. 0:15:44.580 --> 0:15:49.750 What they're most concerned about is straight counselling, and they're, 0:15:49.760 --> 0:15:53.010 you know, understandably concerned about, you know, when you get 0:15:53.020 --> 0:15:57.729 complex case formulations and so forth. And they're worried about 0:15:57.780 --> 0:16:00.669 the kind of regulatory framework and where we're at in 0:16:00.679 --> 0:16:04.960 terms of the regulatory framework. So obviously, as a clinician, 0:16:04.969 --> 0:16:07.760 there's a lot of regulation that goes around being a 0:16:07.770 --> 0:16:10.330 clinician that all still has to be put in place 0:16:10.340 --> 0:16:11.219 for these bots. 0:16:11.229 --> 0:16:14.739 The Therapeutic Goods Administration is the organisation that would be 0:16:14.750 --> 0:16:18.270 relevant for that in this case in Australia. And so 0:16:18.280 --> 0:16:20.580 we're very interested in starting to work with them to 0:16:20.669 --> 0:16:23.299 talk about, you know? Well, how do we go about 0:16:23.309 --> 0:16:24.409 regulating these things? 0:16:25.590 --> 0:16:27.859 One of the things to be clear about here, too, 0:16:27.869 --> 0:16:31.450 is that we're specifically talking about generative AI as opposed 0:16:31.460 --> 0:16:34.849 to AI in general, so generative AI is when you're 0:16:34.859 --> 0:16:40.450 actually creating the texts in this case, kind of fresh 0:16:40.460 --> 0:16:44.340 on every occurrence. Right? So there's a bunch of products 0:16:44.349 --> 0:16:47.099 in the market already where what they're doing is they're 0:16:47.109 --> 0:16:51.130 using AI, but there's a kind of library of possible 0:16:51.140 --> 0:16:53.530 responses and what they're using the AI to do is 0:16:53.539 --> 0:16:57.010 to choose from that library, and those have been demonstrated 0:16:57.020 --> 0:17:00.460 to be quite efficacious. But obviously there's a ceiling to 0:17:00.469 --> 0:17:03.979 that because you're talking to specific individuals. And so it 0:17:03.989 --> 0:17:06.163 doesn't know about that particular person in the way that 0:17:06.173 --> 0:17:09.813 the generative AI does. But there's also a risk, because 0:17:09.823 --> 0:17:12.203 if you've got a library of things you're choosing from, 0:17:12.213 --> 0:17:13.953 you can just go through and make sure every single 0:17:13.963 --> 0:17:16.163 one of those is OK to say. Whereas with the 0:17:16.173 --> 0:17:20.143 generative AI, it's coming up with its own responses. And 0:17:20.154 --> 0:17:22.802 so the there's much more of an onus to make 0:17:22.813 --> 0:17:24.803 sure it's not going to say something it shouldn't 0:17:24.813 --> 0:17:27.292 So when you say it like this, a lot of 0:17:27.303 --> 0:17:31.310 the reservations that practising clinicians seem to have about this 0:17:31.319 --> 0:17:34.089 sort of pragmatic and, you know they they seem to 0:17:34.099 --> 0:17:36.089 be open to it as long as it's mostly doing 0:17:36.099 --> 0:17:38.290 the sort of helper roles rather than the therapy roles. 0:17:38.300 --> 0:17:40.780 I mean, are there any sort of more philosophical objections 0:17:40.790 --> 0:17:42.390 to people often say this is the sort of thing 0:17:42.400 --> 0:17:46.680 which in principle a computer can't do. These are the 0:17:46.689 --> 0:17:49.119 sort of things, you know, that I'm not endorsing this. 0:17:49.130 --> 0:17:51.229 I'm just saying, I'm sure people of my generation would 0:17:51.239 --> 0:17:55.659 say "Therapy is this mysterious connection with another human being 0:17:55.659 --> 0:17:59.849 and you couldn't possibly mechanise it through a computer and surely, 0:17:59.859 --> 0:18:04.219 you know, library or not,"-to use your last metaphor- "You know, 0:18:04.229 --> 0:18:06.319 all the sensitivity, all the reading I've done, all of 0:18:06.329 --> 0:18:09.208 the therapy I've done, all of the deep thought I've 0:18:09.219 --> 0:18:11.800 gone to. Surely that can't be replicated by some sort 0:18:11.810 --> 0:18:14.135 of machine." Do you get that kind of reservation as well? 0:18:14.145 --> 0:18:17.676 Um, yeah, absolutely. We do. Perhaps not quite as much 0:18:17.686 --> 0:18:21.355 as I had anticipated when I first started, like I thought, 0:18:21.475 --> 0:18:23.515 you know, a lot of people would have that reservation, 0:18:23.526 --> 0:18:26.094 but and there's certainly been some, but it hasn't been 0:18:26.105 --> 0:18:29.186 a dominant, um, response. I think it makes a big 0:18:29.196 --> 0:18:31.706 difference once people actually do interact with it and see 0:18:31.715 --> 0:18:35.156 it operating, particularly the newer ones. Yeah, there are just 0:18:35.166 --> 0:18:37.890 some things it does, and it's like, wow, you know, 0:18:37.901 --> 0:18:41.602 that's really, really remarkable. But, you know, by the same token, 0:18:41.612 --> 0:18:46.062 I don't want to give the impression that everything's solved. Right. So, um, 0:18:46.071 --> 0:18:48.401 so we're doing a lot of work at the moment, 0:18:48.552 --> 0:18:52.962 thinking about, you know, what is the clinician doing? So 0:18:52.972 --> 0:18:57.192 there's this whole approach to generative AI called chain-of-thought prompting. 0:18:57.462 --> 0:19:01.501 And so what chain-of-thought prompting does is it says before 0:19:01.511 --> 0:19:05.323 actually producing the response: tell me what you would think. 0:19:05.323 --> 0:19:08.699 And so there's demonstrations that this makes a huge difference 0:19:08.709 --> 0:19:11.550 in terms of performance in general in generative AI. So 0:19:11.560 --> 0:19:14.180 what we've been doing is taking that lesson and saying, OK, 0:19:14.280 --> 0:19:18.010 let's think about well, what's a clinician thinking as they're 0:19:18.020 --> 0:19:21.530 going through to produce a response? And so we're trying 0:19:21.540 --> 0:19:23.599 to build that in, and so now you can kind 0:19:23.609 --> 0:19:26.689 of see some of that, and that definitely increases the 0:19:26.699 --> 0:19:30.349 quality of the responses. But we've still got a long 0:19:30.359 --> 0:19:32.589 ways to go with that. So I think the right 0:19:32.599 --> 0:19:35.800 there's right to be some of that objection, but I 0:19:35.810 --> 0:19:38.989 don't think it's an in principle issue, but there's certainly 0:19:39.000 --> 0:19:40.413 we're not there yet. 0:19:40.413 --> 0:19:42.069 And I think a lot of our listeners would have 0:19:42.079 --> 0:19:45.550 played around with some generative AI and asked questions of 0:19:45.560 --> 0:19:49.079 ChatGPT or whatever model they use and see both some 0:19:49.089 --> 0:19:54.310 really impressive answers, but also some that are just clearly wrong. 0:19:54.869 --> 0:19:58.280 How do you make sure that the model is going 0:19:58.290 --> 0:20:01.839 to say the right thing at the right time to 0:20:01.849 --> 0:20:02.669 a patient? 0:20:02.709 --> 0:20:04.839 So there are different things that can be right and 0:20:04.849 --> 0:20:08.260 wrong about a response. So So the most dangerous thing 0:20:08.270 --> 0:20:13.810 is if it says something which might lead a client 0:20:13.819 --> 0:20:17.290 to do something, you know, dangerous to them, to their health, right? 0:20:18.069 --> 0:20:20.160 And so what we've done is we've developed a whole 0:20:20.170 --> 0:20:24.329 bunch of cases, safety cases. And so whenever before our 0:20:24.339 --> 0:20:27.560 bot goes out, we run all of these safety cases, 0:20:27.569 --> 0:20:30.650 see how it's responding, how the latest version is responding 0:20:30.930 --> 0:20:33.890 and that's all automatically marked. And we go through and 0:20:33.900 --> 0:20:38.020 look at the responses on those to make sure that, um, 0:20:38.160 --> 0:20:41.149