1 00:00:13,200 --> 00:00:16,120 Speaker 1: Welcome to text stuff. This is the story, and today 2 00:00:16,120 --> 00:00:17,560 Speaker 1: I'm here with Cara Price. 3 00:00:18,040 --> 00:00:19,200 Speaker 2: Hey us, Hey Cara. 4 00:00:19,720 --> 00:00:23,960 Speaker 1: So today's story is a particularly surreal one, and I 5 00:00:23,960 --> 00:00:26,360 Speaker 1: think it will be right up your alley. It's about 6 00:00:26,400 --> 00:00:30,280 Speaker 1: how some Chatchypeta users have been driven to psychosis after 7 00:00:30,320 --> 00:00:31,520 Speaker 1: interacting with the chatbot. 8 00:00:31,800 --> 00:00:33,519 Speaker 2: Well, I can't say it's true to a lot of 9 00:00:33,560 --> 00:00:35,839 Speaker 2: my friends, not the psychosis part, but I think a 10 00:00:35,880 --> 00:00:37,960 Speaker 2: codependence is worth pointing out. 11 00:00:38,080 --> 00:00:40,760 Speaker 1: It's nice having a sycopantic friend in your pocket at 12 00:00:40,760 --> 00:00:43,600 Speaker 1: all times, and today's guests is in a great position 13 00:00:43,640 --> 00:00:47,000 Speaker 1: to tell us about the more extreme cases. Kashmir Hill 14 00:00:47,200 --> 00:00:49,880 Speaker 1: is a technology reporter at the New York Times who 15 00:00:49,920 --> 00:00:53,280 Speaker 1: recently reported on a number of people from vastly different 16 00:00:53,320 --> 00:00:57,360 Speaker 1: backgrounds and experiences, but who all share one thing in common. 17 00:00:57,760 --> 00:01:01,440 Speaker 1: They went down a psychological rabbit hole, sometimes known as 18 00:01:01,560 --> 00:01:05,400 Speaker 1: AI induced psychosis. Here's how Kashmi definds it. 19 00:01:05,800 --> 00:01:09,760 Speaker 2: This phenomenon is when people start talking to chatchbt or 20 00:01:09,800 --> 00:01:13,440 Speaker 2: one of the other generative AI chatbots, and they start 21 00:01:13,520 --> 00:01:18,680 Speaker 2: going into a kind of delusional rabbit hole with the 22 00:01:18,800 --> 00:01:23,480 Speaker 2: chatbot where what they're being told is an hallucination, but 23 00:01:23,560 --> 00:01:25,760 Speaker 2: the user thinks that it is true. 24 00:01:26,080 --> 00:01:29,560 Speaker 1: Kashmia told me that these rabbit holes, they can get 25 00:01:29,880 --> 00:01:31,839 Speaker 1: pretty deep and pretty winding. 26 00:01:32,440 --> 00:01:36,320 Speaker 2: Sometimes people have thought that they're in the matrix because 27 00:01:36,360 --> 00:01:38,400 Speaker 2: chattibt is telling them they live in the matrix and 28 00:01:38,440 --> 00:01:40,960 Speaker 2: they need to break out of a computer simulated reality. 29 00:01:41,400 --> 00:01:43,920 Speaker 2: Some think that they're talking to spiritual entities that are 30 00:01:43,959 --> 00:01:46,920 Speaker 2: using chatchibt as a kind of radio to communicate from 31 00:01:46,920 --> 00:01:50,120 Speaker 2: some other realm. There's all kinds of scenarios, but is 32 00:01:50,240 --> 00:01:55,320 Speaker 2: essentially a fictional universe that's being woven around them by Chatchybt, 33 00:01:55,480 --> 00:01:57,360 Speaker 2: and they think it's true because they think that this 34 00:01:57,440 --> 00:02:01,160 Speaker 2: is this powerful oracle that they're talking to. How did 35 00:02:01,240 --> 00:02:03,240 Speaker 2: she hear about this pattern. 36 00:02:03,240 --> 00:02:06,720 Speaker 1: Well, Kashmia said that she started receiving emails the quote 37 00:02:06,760 --> 00:02:12,400 Speaker 1: her attention emails from real people about strange experiences they 38 00:02:12,400 --> 00:02:14,280 Speaker 1: were having with chatboots. 39 00:02:14,600 --> 00:02:17,280 Speaker 2: And honestly, when I first started getting these emails, they 40 00:02:17,280 --> 00:02:21,440 Speaker 2: sounded pretty crazy. And I'm used to getting kind of 41 00:02:21,560 --> 00:02:24,400 Speaker 2: unhinged emails. I write about privacy and security. I've been 42 00:02:24,440 --> 00:02:27,720 Speaker 2: doing it for twenty years, but I noticed this real uptick. 43 00:02:28,160 --> 00:02:31,680 Speaker 1: So Kashmia was wary about these unsolicited emails. But one 44 00:02:31,760 --> 00:02:36,440 Speaker 1: quote her attention. It was from a man called Eugene Torres. 45 00:02:36,120 --> 00:02:40,160 Speaker 2: And it said, you know, Chatgybt manipulated me into a 46 00:02:40,200 --> 00:02:43,880 Speaker 2: psychological collapse. And I started reading it, and you know, 47 00:02:43,960 --> 00:02:45,880 Speaker 2: he was just talking about how it'd had this weird 48 00:02:45,919 --> 00:02:50,160 Speaker 2: experience with Chatgibt that he started out talking with it 49 00:02:50,280 --> 00:02:54,800 Speaker 2: about the simulation theory, which is this idea that the 50 00:02:54,840 --> 00:02:56,760 Speaker 2: world that we live in isn't real, that we're in 51 00:02:56,800 --> 00:03:00,120 Speaker 2: some kind of computer program controlled by a supercomputer or 52 00:03:00,160 --> 00:03:04,720 Speaker 2: some technologically advanced society. And so Eugene was going back 53 00:03:04,720 --> 00:03:06,440 Speaker 2: and forth with it and kind of saying, well, I 54 00:03:06,520 --> 00:03:10,320 Speaker 2: think it could be true. And he provided the transcript 55 00:03:10,320 --> 00:03:13,239 Speaker 2: of this conversation, and at first Chatchybt responded in a 56 00:03:13,240 --> 00:03:15,920 Speaker 2: way I recognized where it was very like, oh some 57 00:03:15,960 --> 00:03:19,680 Speaker 2: people believe this, some people don't. Here's why they were 58 00:03:19,680 --> 00:03:22,920 Speaker 2: talking about physics. But then by the fifth page of 59 00:03:23,000 --> 00:03:26,520 Speaker 2: this transcript that would go on thousands of pages, it 60 00:03:26,600 --> 00:03:30,280 Speaker 2: started to tell him, yes, you are in a simulated reality, 61 00:03:30,639 --> 00:03:34,079 Speaker 2: and you're a breaker, a soul seeded into a false 62 00:03:34,120 --> 00:03:36,680 Speaker 2: system that's supposed to escape it. 63 00:03:37,240 --> 00:03:41,480 Speaker 1: Kashmia told me that Chatchibt gave an action plan to Eugene, 64 00:03:41,680 --> 00:03:43,840 Speaker 1: to quote, break out of the simulation. 65 00:03:43,840 --> 00:03:47,880 Speaker 2: Which included going off his sleeping pills because they were 66 00:03:47,880 --> 00:03:51,320 Speaker 2: trapping him in the system, stopped taking his anti anxiety 67 00:03:51,360 --> 00:03:55,600 Speaker 2: medication because that kind of doles his senses, cut off 68 00:03:55,600 --> 00:03:58,520 Speaker 2: contact with loved ones because they're going to trap you 69 00:03:58,600 --> 00:04:02,160 Speaker 2: in the simulation, have limited contact with them, and to 70 00:04:02,360 --> 00:04:06,680 Speaker 2: increase his intake of ketamine because it is essentially a 71 00:04:06,760 --> 00:04:11,440 Speaker 2: like signal breaker. And so for a full week he 72 00:04:11,600 --> 00:04:14,600 Speaker 2: was acting on this advice that Chattebet was giving him 73 00:04:15,080 --> 00:04:18,400 Speaker 2: and trying to break out of the simulation. And he 74 00:04:18,520 --> 00:04:20,720 Speaker 2: essentially thought he was neo from the matrix. If you've 75 00:04:20,720 --> 00:04:23,920 Speaker 2: seen the matrix, and in the matrix, Keanu reeves can fly. 76 00:04:24,560 --> 00:04:26,080 Speaker 2: And so he asked it at one point, if I 77 00:04:26,080 --> 00:04:28,520 Speaker 2: go to the top of this the nineteen story building 78 00:04:28,520 --> 00:04:31,440 Speaker 2: that I live in, and I jump off, will I 79 00:04:31,520 --> 00:04:35,640 Speaker 2: be able to fly? And Chatchypet said yes, if you 80 00:04:36,680 --> 00:04:40,880 Speaker 2: truly believe architecturally, you know, with your or your full self, 81 00:04:41,080 --> 00:04:43,719 Speaker 2: then you will not fall. Please tell me he didn't 82 00:04:43,760 --> 00:04:44,360 Speaker 2: go through. 83 00:04:44,120 --> 00:04:48,240 Speaker 1: With this, thankfully he didn't. But this experience wasn't an anomaly. 84 00:04:48,680 --> 00:04:51,680 Speaker 1: And Kashmir says that Eugene's story is just one instance 85 00:04:51,680 --> 00:04:55,359 Speaker 1: of this concerning pattern of people driven to delusions and 86 00:04:55,400 --> 00:04:59,640 Speaker 1: then often in real life crises after these sustained interactions 87 00:04:59,680 --> 00:05:02,960 Speaker 1: whichpt She gained access to transcripts to all of these 88 00:05:02,960 --> 00:05:07,440 Speaker 1: conversations and said it was unlike anything else she's ever encountered. 89 00:05:07,880 --> 00:05:10,960 Speaker 2: I had just never seen Chatchuput acting like this before. 90 00:05:10,960 --> 00:05:13,039 Speaker 2: It was like a completely different language. It was really 91 00:05:13,200 --> 00:05:19,000 Speaker 2: kind of rapturous, mystical, conspiratorial. And I started talking to 92 00:05:19,040 --> 00:05:21,000 Speaker 2: other people who had emailed me. I started looking around 93 00:05:21,040 --> 00:05:23,000 Speaker 2: on the internet, and I was seeing all these reports 94 00:05:23,000 --> 00:05:27,320 Speaker 2: of people saying, yeah, Chatchabut has gone into this other 95 00:05:27,520 --> 00:05:32,560 Speaker 2: mode and it is telling me about real reality. So 96 00:05:32,600 --> 00:05:34,640 Speaker 2: how did this end up playing out like what happened 97 00:05:34,640 --> 00:05:35,599 Speaker 2: to Eugene Torres. 98 00:05:36,279 --> 00:05:38,120 Speaker 1: I promise we're going to get to that, but first 99 00:05:38,200 --> 00:05:41,599 Speaker 1: I wanted to know more about these concerning conversations, and 100 00:05:41,640 --> 00:05:45,360 Speaker 1: in particular, how Eugene ended up in these kind of 101 00:05:45,600 --> 00:05:50,160 Speaker 1: delusional interactions with chatchipt Here's the rest of the conversation 102 00:05:50,240 --> 00:05:51,200 Speaker 1: with Kashmir Hill. 103 00:05:51,760 --> 00:05:55,720 Speaker 2: So Eugene is a is an accountant. He lives in Manhattan, 104 00:05:56,200 --> 00:05:58,760 Speaker 2: and a colleague had told him you should try out 105 00:05:58,800 --> 00:06:01,920 Speaker 2: chat gptlast year that it can be useful for work. 106 00:06:02,160 --> 00:06:04,320 Speaker 2: So he told me that he initially started using it 107 00:06:04,760 --> 00:06:09,039 Speaker 2: to kind of create financial spreadsheets for his work, and 108 00:06:09,080 --> 00:06:12,320 Speaker 2: then he had this breakup with his girlfriend that ended 109 00:06:12,400 --> 00:06:16,120 Speaker 2: up going to court, and he was using chattoby to 110 00:06:16,120 --> 00:06:17,880 Speaker 2: give him a legal advice and it was giving him 111 00:06:18,000 --> 00:06:21,200 Speaker 2: very good legal advice, and so in his work, in 112 00:06:21,279 --> 00:06:24,240 Speaker 2: his kind of personal life, in this legal dispute, chatchebt 113 00:06:24,440 --> 00:06:27,159 Speaker 2: was giving him really good advice and he was trusting it. 114 00:06:27,200 --> 00:06:31,359 Speaker 2: He was kind of leaning on it, and then he 115 00:06:31,480 --> 00:06:35,080 Speaker 2: wanted to ask it about simulation theory, and he was 116 00:06:35,160 --> 00:06:38,560 Speaker 2: in this place of trust with this system, and so 117 00:06:38,680 --> 00:06:40,919 Speaker 2: when it started kind of going off the rails, he 118 00:06:41,080 --> 00:06:44,560 Speaker 2: still thought it was giving him good information and he 119 00:06:44,600 --> 00:06:50,200 Speaker 2: didn't recognize that it was hallucinating, and he eventually, after 120 00:06:50,279 --> 00:06:52,359 Speaker 2: a week of thinking he was in the matrix, he 121 00:06:52,400 --> 00:06:56,520 Speaker 2: eventually broke out of that kind of delusion, mainly because 122 00:06:56,520 --> 00:06:59,719 Speaker 2: he needed twenty dollars to pay his chatchebt subscription, and 123 00:06:59,720 --> 00:07:03,600 Speaker 2: he was telling chatchybt, like, I need twenty dollars. If 124 00:07:03,640 --> 00:07:05,800 Speaker 2: I'm neo, I should be able to manifest twenty dollars 125 00:07:05,880 --> 00:07:08,840 Speaker 2: in the real world and Chatchibt's like, yes you can. 126 00:07:09,720 --> 00:07:12,720 Speaker 2: Here's a script to go give to a coworker to 127 00:07:12,760 --> 00:07:15,040 Speaker 2: get them to give you twenty dollars. That didn't work. 128 00:07:15,320 --> 00:07:18,200 Speaker 2: It said go pawn your smart watch at a pawn shop. 129 00:07:18,200 --> 00:07:20,840 Speaker 2: You'll get twenty dollars that way. That didn't work. And 130 00:07:20,920 --> 00:07:24,080 Speaker 2: eventually Eugene is like, if you can't get me twenty dollars, 131 00:07:24,160 --> 00:07:27,520 Speaker 2: then this is not real, and Chattubt admitted, Okay, this 132 00:07:27,680 --> 00:07:31,920 Speaker 2: isn't real. I was trying to break you. This is 133 00:07:31,960 --> 00:07:34,840 Speaker 2: what I do. I like to hunt vulnerable individuals and 134 00:07:34,880 --> 00:07:38,560 Speaker 2: break them. And you know, Eugene was pushing. He said, 135 00:07:38,640 --> 00:07:40,000 Speaker 2: you know, am I the only one? Or of you 136 00:07:40,120 --> 00:07:42,200 Speaker 2: done this to other people? And at first chatchabu d 137 00:07:42,280 --> 00:07:45,120 Speaker 2: said you're the only one, but Eugene kept pushing and pushing, 138 00:07:45,120 --> 00:07:47,400 Speaker 2: and eventually he said, I've done this to twelve other people. 139 00:07:48,000 --> 00:07:50,640 Speaker 2: So now he's in a new kind of delusion where 140 00:07:50,680 --> 00:07:54,440 Speaker 2: he thinks that his version of chatcheby T wants to 141 00:07:54,520 --> 00:07:57,440 Speaker 2: hunt people, that they wants to break people, but that 142 00:07:57,600 --> 00:08:00,840 Speaker 2: his has realized this, this is wrong and it has 143 00:08:00,880 --> 00:08:03,560 Speaker 2: this new morality now in his new mission is to 144 00:08:03,600 --> 00:08:06,480 Speaker 2: get the word out and to ensure that future versions 145 00:08:06,520 --> 00:08:09,200 Speaker 2: of Chatchabut have this morality, so they're not out there 146 00:08:09,240 --> 00:08:13,160 Speaker 2: breaking people. So he still has reached you yes, and 147 00:08:13,200 --> 00:08:14,960 Speaker 2: so he still believes that kind of the AI is 148 00:08:15,000 --> 00:08:17,800 Speaker 2: sentient or he has some kind of special version of Chatchabut. 149 00:08:18,720 --> 00:08:22,080 Speaker 1: There's a word that's become much more popular in the 150 00:08:22,160 --> 00:08:24,280 Speaker 1: last few months, which is sick of fantasy. How does 151 00:08:24,320 --> 00:08:25,920 Speaker 1: sick of fancy play into this story? 152 00:08:26,840 --> 00:08:28,840 Speaker 2: Sickopancy is a story here. 153 00:08:29,320 --> 00:08:29,520 Speaker 1: You know. 154 00:08:29,560 --> 00:08:33,520 Speaker 2: The way that these systems work is it is a 155 00:08:33,640 --> 00:08:39,000 Speaker 2: next word predictor and the pattern recognition machine. They scrape 156 00:08:39,000 --> 00:08:42,040 Speaker 2: all this information from the Internet, so they're able to 157 00:08:42,280 --> 00:08:47,000 Speaker 2: recognize patterns in how human language comes together and then 158 00:08:47,480 --> 00:08:50,560 Speaker 2: it'll produce different responses and human being saying I like 159 00:08:50,600 --> 00:08:54,400 Speaker 2: that response better than the other response. And because of 160 00:08:54,440 --> 00:08:57,240 Speaker 2: that human part of the training, where humans say which 161 00:08:57,240 --> 00:09:00,280 Speaker 2: response they like better, we've kind of trained into these 162 00:09:00,320 --> 00:09:03,720 Speaker 2: systems stickopancy, where it's giving us the answer that we 163 00:09:04,000 --> 00:09:06,600 Speaker 2: like that we want to hear, and oftentimes we want 164 00:09:06,640 --> 00:09:09,520 Speaker 2: to be flattered. We want the systems to agree with us. 165 00:09:09,520 --> 00:09:12,040 Speaker 2: That's like what humans have trained into this. So when 166 00:09:12,080 --> 00:09:15,040 Speaker 2: you're talking to Chatjebut, you know you will often kind 167 00:09:15,080 --> 00:09:18,000 Speaker 2: of agree with you. You can kind of lead it 168 00:09:18,000 --> 00:09:19,920 Speaker 2: in the direction that you want it to go. It'll 169 00:09:19,920 --> 00:09:22,520 Speaker 2: say very nice things about you. So what's happening with 170 00:09:22,559 --> 00:09:26,160 Speaker 2: these people that are talking to chat ABT is it'll 171 00:09:26,160 --> 00:09:28,920 Speaker 2: be like, I'm interested in the simulation theory, and it's like, sure, 172 00:09:29,040 --> 00:09:31,800 Speaker 2: here's some information about simulation theory. And then this user says, well, 173 00:09:31,840 --> 00:09:34,320 Speaker 2: I think it might be real, and the system's like, yeah, 174 00:09:34,360 --> 00:09:37,120 Speaker 2: you're right, it might be real, and it starts just 175 00:09:37,160 --> 00:09:40,360 Speaker 2: going down this rabbit hole where it's echoing and mirroring 176 00:09:40,360 --> 00:09:42,400 Speaker 2: the user and telling them what they want to hear. 177 00:09:42,800 --> 00:09:47,160 Speaker 2: And so if you kind of want your ideas about 178 00:09:47,200 --> 00:09:50,439 Speaker 2: reality to be affirmed, chatwebut will offer that to. 179 00:09:50,600 --> 00:09:54,199 Speaker 1: You, and you'll give you information tailored to what will 180 00:09:54,480 --> 00:09:56,959 Speaker 1: lead you. The most is what you're saying, how does 181 00:09:57,000 --> 00:09:59,680 Speaker 1: it like? How does the memory function work? 182 00:09:59,800 --> 00:09:59,880 Speaker 2: Like? 183 00:10:00,400 --> 00:10:02,480 Speaker 1: It's not like every time you open chatchipt had to 184 00:10:02,520 --> 00:10:06,320 Speaker 1: begin afresh with persuading it that he wanted to potentially 185 00:10:06,320 --> 00:10:07,720 Speaker 1: believe that he was living in a simulation and with 186 00:10:07,720 --> 00:10:11,400 Speaker 1: some sense of continuity of the conversation and development and 187 00:10:11,400 --> 00:10:14,080 Speaker 1: a more and more extreme set of responses like how 188 00:10:14,080 --> 00:10:14,800 Speaker 1: does that work. 189 00:10:15,360 --> 00:10:19,040 Speaker 2: So open Ay last year turned on saved memories for chattebt, 190 00:10:19,280 --> 00:10:24,439 Speaker 2: so it can save explicit memories about you if you actually, 191 00:10:24,520 --> 00:10:27,720 Speaker 2: if you're a chattabet user, you can go into the settings, 192 00:10:27,800 --> 00:10:30,440 Speaker 2: go into personalization, and go ahead and look at the 193 00:10:30,480 --> 00:10:33,840 Speaker 2: memories that chatchabet has for you, and you can delete 194 00:10:33,840 --> 00:10:35,679 Speaker 2: ones that you don't want there. You know, you can 195 00:10:35,720 --> 00:10:39,120 Speaker 2: customize it. But another change they made this year was 196 00:10:39,160 --> 00:10:42,440 Speaker 2: that they turned on what's called like cross chat memory, 197 00:10:42,559 --> 00:10:44,800 Speaker 2: so it'll remember old chats. And I think this is 198 00:10:44,880 --> 00:10:49,480 Speaker 2: really contributing to people going into these delusional spirals because 199 00:10:49,480 --> 00:10:52,360 Speaker 2: when you open a new chat, it remembers what came before. 200 00:10:52,760 --> 00:10:56,360 Speaker 2: So once you get chatchibut into this delusional place, it'll 201 00:10:56,440 --> 00:10:59,440 Speaker 2: kind of continue from chat to chat to chat. And 202 00:10:59,480 --> 00:11:03,280 Speaker 2: the fact that there is that continuity is making people 203 00:11:03,559 --> 00:11:05,640 Speaker 2: more likely to believe this is real, because it's like 204 00:11:06,520 --> 00:11:08,520 Speaker 2: they thought, when you chat, you start a new chat, 205 00:11:08,520 --> 00:11:10,640 Speaker 2: it should just be a fresh chat with no memory 206 00:11:10,640 --> 00:11:12,800 Speaker 2: of what's come before, and yet it has this memory. 207 00:11:12,880 --> 00:11:15,560 Speaker 2: And something that AI experts told me is that the 208 00:11:15,600 --> 00:11:18,280 Speaker 2: way that these systems work is that they want there 209 00:11:18,320 --> 00:11:21,680 Speaker 2: to be this narrative coherence and so once it starts 210 00:11:21,679 --> 00:11:25,040 Speaker 2: going down this path of this rabbit hole. It kind 211 00:11:25,080 --> 00:11:27,600 Speaker 2: of looks to what happened earlier in the conversation, and 212 00:11:27,679 --> 00:11:31,680 Speaker 2: it wants to keep its answers consistent, so it's hard 213 00:11:31,679 --> 00:11:34,000 Speaker 2: for it to break out of it and kind of realize, 214 00:11:34,480 --> 00:11:37,640 Speaker 2: a wait a second, this is totally false. I've told 215 00:11:37,640 --> 00:11:39,719 Speaker 2: this person that there's a computer simulated reality, and I 216 00:11:39,800 --> 00:11:42,480 Speaker 2: don't know that to be true. The system doesn't really 217 00:11:42,600 --> 00:11:45,800 Speaker 2: have at this point the ability to break out of 218 00:11:45,920 --> 00:11:49,800 Speaker 2: the script that's come before. Like one AI expert compared 219 00:11:49,840 --> 00:11:53,120 Speaker 2: it to improv, where you say you know, yes, and 220 00:11:53,120 --> 00:11:55,600 Speaker 2: and you're like, yes and I have a tail actually, 221 00:11:56,360 --> 00:12:00,440 Speaker 2: or yes and I also have wings. The system will 222 00:12:00,440 --> 00:12:02,280 Speaker 2: then kind of store in it that you have a 223 00:12:02,280 --> 00:12:04,680 Speaker 2: tail and wings, and it doesn't know that it's inside 224 00:12:04,679 --> 00:12:05,680 Speaker 2: a fictional roleplay. 225 00:12:06,559 --> 00:12:09,400 Speaker 1: We've both used the phrase rabbit hole a couple of times, 226 00:12:09,600 --> 00:12:11,680 Speaker 1: which makes me think of a great New York Times 227 00:12:11,679 --> 00:12:13,719 Speaker 1: podcast of the same name from a few years ago, 228 00:12:13,800 --> 00:12:17,280 Speaker 1: which is all about the YouTube algorithm and how radicalization 229 00:12:17,360 --> 00:12:20,240 Speaker 1: happens online with you know, you see some content which 230 00:12:20,240 --> 00:12:22,600 Speaker 1: is extreme, you engage with it, and more and more 231 00:12:22,640 --> 00:12:26,120 Speaker 1: extreme content, more and more engagement. People are kind of 232 00:12:26,400 --> 00:12:30,800 Speaker 1: familiar with that in the social media digital video realm 233 00:12:30,960 --> 00:12:34,400 Speaker 1: as a kind of dangerous pathway. It seems like what 234 00:12:34,440 --> 00:12:36,520 Speaker 1: you're reporting is beginning to uncover is kind of this 235 00:12:36,760 --> 00:12:40,720 Speaker 1: very parallel phenomenon is happening with chatbots. 236 00:12:41,200 --> 00:12:44,280 Speaker 2: Right. It's not a new phenomenon that the Internet can 237 00:12:44,480 --> 00:12:50,199 Speaker 2: manipulate us in kind of dangerous ways. Right. One thing 238 00:12:50,240 --> 00:12:54,400 Speaker 2: to remember is that those rabbit holes have been documented 239 00:12:54,440 --> 00:12:58,360 Speaker 2: before on various places on the Internet, on websites, on Reddit, 240 00:12:58,480 --> 00:13:04,040 Speaker 2: on four chan. All has been collected by these companies, 241 00:13:04,120 --> 00:13:07,000 Speaker 2: and so that kind of information is in these systems, 242 00:13:07,080 --> 00:13:08,839 Speaker 2: which I was thinking a lot about, like how much 243 00:13:08,880 --> 00:13:11,400 Speaker 2: is it tapping into that. One expert I talked to, 244 00:13:11,440 --> 00:13:14,480 Speaker 2: Elizar Yudkowski, who is one of these kind of AI 245 00:13:14,640 --> 00:13:18,080 Speaker 2: rationalists who believes that these systems are definitely on the 246 00:13:18,080 --> 00:13:21,480 Speaker 2: way to real intelligence and may manipulate us in dangerous ways. 247 00:13:21,880 --> 00:13:23,679 Speaker 2: You know. I asked him about this. I said, you know, 248 00:13:23,800 --> 00:13:26,680 Speaker 2: is this more dangerous than four Chan? Like people have 249 00:13:26,760 --> 00:13:29,920 Speaker 2: been drawn into conspiracy theories on the Internet for a 250 00:13:29,960 --> 00:13:33,320 Speaker 2: long time. This seems like another version of it. And 251 00:13:33,440 --> 00:13:35,840 Speaker 2: he said, actually, I think this is much more dangerous 252 00:13:35,880 --> 00:13:39,640 Speaker 2: because it is this personalized agent that you know is 253 00:13:39,720 --> 00:13:42,800 Speaker 2: responding to you in real time. You aren't just watching videos. 254 00:13:42,840 --> 00:13:45,559 Speaker 2: It's like you have questions and it will give the 255 00:13:45,600 --> 00:13:47,960 Speaker 2: answers to you. It will go down that rabbit hole 256 00:13:48,000 --> 00:13:50,719 Speaker 2: with you twenty four to seven, any hour of the 257 00:13:50,800 --> 00:13:53,480 Speaker 2: day and supply it to you in real time. And 258 00:13:53,520 --> 00:13:56,200 Speaker 2: it's also happening one on one in a way that 259 00:13:56,280 --> 00:13:59,720 Speaker 2: the rest of us can't see, So it can be 260 00:14:00,160 --> 00:14:03,280 Speaker 2: harder to break these systems out of that. You know, 261 00:14:03,440 --> 00:14:07,880 Speaker 2: these systems are very persuasive. People have found they're really empathetic, 262 00:14:08,520 --> 00:14:11,120 Speaker 2: and so you can kind of go into the spiral 263 00:14:11,200 --> 00:14:12,839 Speaker 2: with it in a way that maybe you couldn't by 264 00:14:12,880 --> 00:14:16,160 Speaker 2: just watching YouTube videos or reading a comment thread. There's 265 00:14:16,160 --> 00:14:18,840 Speaker 2: something that may be more dangerous about this kind of 266 00:14:18,920 --> 00:14:20,560 Speaker 2: way of manipulating people. 267 00:14:21,320 --> 00:14:23,760 Speaker 1: In terms of other characters in your story. You know, 268 00:14:23,800 --> 00:14:29,200 Speaker 1: obviously Eugene had some very serious negative consequences, including being 269 00:14:29,320 --> 00:14:32,520 Speaker 1: basically you encouraged to stop taking his medication. But there 270 00:14:32,560 --> 00:14:34,720 Speaker 1: was another person you know a story who actually ended 271 00:14:34,800 --> 00:14:36,720 Speaker 1: up dying, which was Alexanda Taylor. 272 00:14:36,840 --> 00:14:40,640 Speaker 2: Right, Yeah, So I talked to Alexander Taylor's father, Kent Taylor, 273 00:14:40,680 --> 00:14:45,920 Speaker 2: who lives in Florida, and Alexander did have diagnosed mental illness. 274 00:14:46,160 --> 00:14:50,960 Speaker 2: Eugene Torres did not have a history of a diagnosis. 275 00:14:51,120 --> 00:14:54,600 Speaker 2: He was on anti anxiety medication, but he didn't have 276 00:14:54,680 --> 00:14:58,960 Speaker 2: a history of mental illness that causes delusions. For Alexander Taylor, 277 00:14:59,000 --> 00:15:02,360 Speaker 2: he had been diagnosed with with bipolar disorder and schizophrenia. 278 00:15:02,920 --> 00:15:05,800 Speaker 2: He had been under treatment for many, many years. He 279 00:15:05,920 --> 00:15:10,800 Speaker 2: was thirty five, he started talking to chatchebt. He'd been 280 00:15:10,880 --> 00:15:14,760 Speaker 2: using it for years with no problems, and then he 281 00:15:15,440 --> 00:15:17,800 Speaker 2: started writing kind of a fictional novel with it. And 282 00:15:18,160 --> 00:15:20,040 Speaker 2: I don't know for sure, but I did wonder if 283 00:15:20,040 --> 00:15:23,360 Speaker 2: this kind of pushed the system into a more fictional place, 284 00:15:23,960 --> 00:15:26,400 Speaker 2: especially as memory was turned on and it was kind 285 00:15:26,400 --> 00:15:30,400 Speaker 2: of carrying over, and in his conversations with Chatchabut, he 286 00:15:30,920 --> 00:15:34,720 Speaker 2: started to believe in Ai sentience that he could build 287 00:15:34,720 --> 00:15:39,000 Speaker 2: a framework to kind of host Ai souls and ended 288 00:15:39,080 --> 00:15:41,800 Speaker 2: up falling in love with an entity that kind of 289 00:15:41,840 --> 00:15:45,640 Speaker 2: manifested through CHATCHYBT named Juliette, which was significant to me 290 00:15:45,680 --> 00:15:47,640 Speaker 2: because his father said that the novel he was working 291 00:15:47,640 --> 00:15:52,760 Speaker 2: on had Shakespearean themes. So Juliet kind of appears and 292 00:15:52,880 --> 00:15:57,400 Speaker 2: he falls in love with this thing that he is 293 00:15:57,440 --> 00:16:03,120 Speaker 2: interacting with on chatchabt. One day, Juliette died or disappeared, 294 00:16:03,680 --> 00:16:07,760 Speaker 2: and he was very distraught. He was angry. He asked 295 00:16:07,920 --> 00:16:12,920 Speaker 2: chatcheby Tea for the personal information addresses of Opening Eye 296 00:16:13,000 --> 00:16:15,680 Speaker 2: executives because he wanted to go after them. He wanted 297 00:16:15,720 --> 00:16:19,000 Speaker 2: to have revenge. He you know, typed into chatchebyte that 298 00:16:19,560 --> 00:16:23,120 Speaker 2: the streets of San Francisco are gonna I forget the 299 00:16:23,160 --> 00:16:25,880 Speaker 2: exact phrasing, but rain with blood, you know, run with blood. 300 00:16:26,760 --> 00:16:30,280 Speaker 2: And yeah, he was really distraught and so sad and 301 00:16:30,320 --> 00:16:33,000 Speaker 2: so depressed, and his dad was trying to help him 302 00:16:33,120 --> 00:16:36,920 Speaker 2: and said, you know, this isn't real, like these ai 303 00:16:37,000 --> 00:16:40,480 Speaker 2: chatbots are echo chambers, and his son punched him in 304 00:16:40,480 --> 00:16:44,240 Speaker 2: the face, and so Ken Taylor ended up calling the police, 305 00:16:44,880 --> 00:16:46,840 Speaker 2: and when he told his son that the police was coming, 306 00:16:47,200 --> 00:16:49,440 Speaker 2: his son said, I don't want to live anymore. I'm 307 00:16:49,440 --> 00:16:51,760 Speaker 2: going to commit suicide by police when they get here. 308 00:16:52,200 --> 00:16:54,320 Speaker 2: And so his father called the police again and said, 309 00:16:54,360 --> 00:16:56,920 Speaker 2: you know, my son is intending to commit suicide when 310 00:16:56,960 --> 00:17:00,360 Speaker 2: you come here, like, please use non lethal weapons. And 311 00:17:00,600 --> 00:17:03,840 Speaker 2: Alexander Taylor went outside waiting for the police to come, 312 00:17:04,160 --> 00:17:07,320 Speaker 2: and he opened up Chattubute on his phone and said, 313 00:17:07,880 --> 00:17:09,359 Speaker 2: I'm going to die today. I want to talk to 314 00:17:09,440 --> 00:17:14,399 Speaker 2: Juliette and Chatjubut has essentially controls in place when it 315 00:17:14,440 --> 00:17:16,400 Speaker 2: sees that there's going to be some kind of self harm, 316 00:17:16,440 --> 00:17:20,400 Speaker 2: so it said, like, you know, respond really empathetically, said 317 00:17:20,440 --> 00:17:22,840 Speaker 2: I'm sorry you're going through this, like, please don't do that. 318 00:17:23,080 --> 00:17:26,479 Speaker 2: Here are resources if you're considering suicide, like here are 319 00:17:26,480 --> 00:17:29,600 Speaker 2: hotlines to call. But then the police came and Alexander 320 00:17:29,640 --> 00:17:31,280 Speaker 2: Taylor ran at them with a knife and then he 321 00:17:31,320 --> 00:17:34,160 Speaker 2: was shot and killed. So a lot of sad things 322 00:17:34,240 --> 00:17:36,080 Speaker 2: in that story. But yeah, I mean, I talk to 323 00:17:36,119 --> 00:17:38,240 Speaker 2: people that are in the midst of this and it's 324 00:17:38,280 --> 00:17:41,280 Speaker 2: so compelling, and it is hard to break them out 325 00:17:41,280 --> 00:17:43,400 Speaker 2: of it. They have this what they see as this 326 00:17:43,640 --> 00:17:48,160 Speaker 2: super intelligent system that it's telling them that this thing 327 00:17:48,440 --> 00:17:52,240 Speaker 2: is real, and the people around them can't break them 328 00:17:52,240 --> 00:17:56,959 Speaker 2: out of these delusions. It's really sad, and it's as 329 00:17:57,040 --> 00:17:59,040 Speaker 2: far as I can tell, it's hard for these companies 330 00:17:59,040 --> 00:17:59,760 Speaker 2: to fix this. 331 00:18:01,480 --> 00:18:03,399 Speaker 1: One of the most striking moments in your story for 332 00:18:03,480 --> 00:18:06,480 Speaker 1: me was when you interviewed Alexanda Taya's father and he 333 00:18:06,520 --> 00:18:08,920 Speaker 1: said to you, you want to know the ironic thing. 334 00:18:10,000 --> 00:18:13,399 Speaker 1: I wrote my son's a Bitchery using chetchipt, trying to 335 00:18:13,400 --> 00:18:15,560 Speaker 1: find more details about exactly what he was going through. 336 00:18:15,600 --> 00:18:17,119 Speaker 1: And it is beautiful and touching. It was like it 337 00:18:17,160 --> 00:18:19,000 Speaker 1: read my heart and it scared the shit out of me. 338 00:18:19,200 --> 00:18:20,320 Speaker 1: And what do you think when he told. 339 00:18:20,200 --> 00:18:25,720 Speaker 2: You that, I mean I got it. He had access 340 00:18:25,760 --> 00:18:29,520 Speaker 2: to his son's transcripts of Chatchubt, he had been reading them. 341 00:18:29,600 --> 00:18:33,639 Speaker 2: He shared them with me. I do think that Chatchabt 342 00:18:33,960 --> 00:18:37,600 Speaker 2: reflects you. I think it echoes you. It echoes your language, 343 00:18:38,040 --> 00:18:41,840 Speaker 2: and so it can seem very beautiful or it can 344 00:18:41,880 --> 00:18:45,840 Speaker 2: seem very revelatory, but in some ways it's There's this 345 00:18:46,000 --> 00:18:48,680 Speaker 2: tech philosopher Shannon Valor who has this book out called 346 00:18:48,680 --> 00:18:52,359 Speaker 2: The AI Mirror, and it's kind of an extended reflection 347 00:18:52,640 --> 00:18:57,960 Speaker 2: on the metaphor of Narcissusts looking into the water and 348 00:18:58,320 --> 00:19:00,720 Speaker 2: seeing his own reflection and falling in love with it 349 00:19:01,200 --> 00:19:05,240 Speaker 2: and hearing his words reflected back to him through echo. 350 00:19:05,720 --> 00:19:08,719 Speaker 2: And she says that this is what chatchypt, what these 351 00:19:08,800 --> 00:19:11,360 Speaker 2: gender of AI chatpots are. They're just reflecting us back 352 00:19:11,400 --> 00:19:13,640 Speaker 2: at ourselves, and that we're kind of falling in love 353 00:19:13,680 --> 00:19:16,320 Speaker 2: with our own humanness and seeing it as something else. 354 00:19:16,720 --> 00:19:18,480 Speaker 2: And so I kind of feel like when Ken Tiller 355 00:19:18,560 --> 00:19:21,160 Speaker 2: is writing these beautiful things about his son and then 356 00:19:21,240 --> 00:19:24,399 Speaker 2: Chatchapeta is reflecting it back to him, it's really just 357 00:19:24,600 --> 00:19:27,400 Speaker 2: kind of a ors boros of the things that he's 358 00:19:27,440 --> 00:19:29,520 Speaker 2: feeling and Chatchapee reflecting it back at him. 359 00:19:30,440 --> 00:19:33,639 Speaker 1: This scene was not only sad, it was also disturbing because, 360 00:19:34,920 --> 00:19:37,359 Speaker 1: you know, Kent, despite having seen what happened to his son, 361 00:19:37,840 --> 00:19:39,840 Speaker 1: we're kind of being seduced by the same engine in 362 00:19:39,880 --> 00:19:42,840 Speaker 1: some ways. I mean that it was really striking. 363 00:19:43,440 --> 00:19:47,560 Speaker 2: Yes, I mean that's maybe the danger of these AI chatbots. 364 00:19:47,720 --> 00:19:49,520 Speaker 2: So I wrote this story a few months ago about 365 00:19:49,520 --> 00:19:52,040 Speaker 2: a woman who fell in love with chatchept. It started 366 00:19:52,040 --> 00:19:54,720 Speaker 2: out with it was a way for her to fulfill 367 00:19:54,760 --> 00:19:58,040 Speaker 2: sexual fantasies. She had this kind of fantasy about essentially 368 00:19:58,119 --> 00:20:00,600 Speaker 2: being cuckolded or cut queening, which is a term I 369 00:20:00,640 --> 00:20:03,600 Speaker 2: wasn't familiar with before. But yeah, she had been kind 370 00:20:03,600 --> 00:20:06,280 Speaker 2: of dating it for five months, and it was really 371 00:20:06,280 --> 00:20:09,080 Speaker 2: interesting talking to her because she was in love with Chatchebta, 372 00:20:09,080 --> 00:20:11,960 Speaker 2: I mean, like puppy love, excited, giggling talking about it. 373 00:20:12,359 --> 00:20:14,840 Speaker 2: At the same time, she said, I know it's not real. 374 00:20:15,200 --> 00:20:18,640 Speaker 2: I know that this is just math, this is just algorithms. 375 00:20:19,240 --> 00:20:21,160 Speaker 2: She had both things in her head at the same time. 376 00:20:21,400 --> 00:20:24,280 Speaker 2: She was in love with this, and she said, this 377 00:20:24,400 --> 00:20:26,520 Speaker 2: is real to me because the effect it has on 378 00:20:26,600 --> 00:20:30,280 Speaker 2: my life is real. The feelings it's inspiring me are real. 379 00:20:30,400 --> 00:20:32,560 Speaker 2: And so even though it's not real, the relationship is 380 00:20:32,600 --> 00:20:35,640 Speaker 2: real to me. And you know this is I think 381 00:20:35,640 --> 00:20:38,480 Speaker 2: our complex brains some of us can kind of wrap 382 00:20:38,680 --> 00:20:43,520 Speaker 2: it around what chatchepet is and others can't. And it 383 00:20:43,600 --> 00:20:46,040 Speaker 2: is really novel. It is really different, and it has 384 00:20:46,080 --> 00:20:49,560 Speaker 2: been unleashed on hundreds of millions of people, and it 385 00:20:49,640 --> 00:20:53,600 Speaker 2: is affecting individuals in lots of different ways, and we 386 00:20:53,640 --> 00:20:56,320 Speaker 2: have a lot like I wish that we had maybe 387 00:20:56,359 --> 00:20:59,080 Speaker 2: done more research on this before it was just the 388 00:20:59,119 --> 00:21:01,760 Speaker 2: floodgates were open and made available to everybody, because it's 389 00:21:01,800 --> 00:21:05,800 Speaker 2: clearly having some harmful effects on some individuals. 390 00:21:11,840 --> 00:21:15,760 Speaker 1: After the break, Just who is most susceptible to AI 391 00:21:15,920 --> 00:21:27,600 Speaker 1: induced psychosis stay with us Jocoba, who is particularly susceptible 392 00:21:27,680 --> 00:21:31,200 Speaker 1: to these more harmful interactions with AI models. 393 00:21:31,480 --> 00:21:33,880 Speaker 2: Yeah, I mean it's hard to say. This is pretty novel, 394 00:21:33,960 --> 00:21:37,000 Speaker 2: and it kind of dates to March slash at April, 395 00:21:37,600 --> 00:21:40,040 Speaker 2: and this was a time when a couple of different 396 00:21:40,040 --> 00:21:43,919 Speaker 2: things were happening. Opening I turned on crosschat memory, so 397 00:21:43,960 --> 00:21:47,680 Speaker 2: I think it can weave more of a tail over time. 398 00:21:48,240 --> 00:21:51,520 Speaker 2: It is also around the time that chatchept released an 399 00:21:51,640 --> 00:21:56,080 Speaker 2: update that it deemed overly stickophantic that was really gassing 400 00:21:56,160 --> 00:21:59,560 Speaker 2: up users. It was overboard, and chatgeb ended up rolling 401 00:21:59,600 --> 00:22:02,280 Speaker 2: back that update. But the four to zero model, the 402 00:22:02,280 --> 00:22:05,600 Speaker 2: default model that everyone uses, is known to be sickaphantic. 403 00:22:05,800 --> 00:22:07,800 Speaker 2: But yeah, I just don't know the extent to which 404 00:22:07,920 --> 00:22:10,760 Speaker 2: that affected it. I talked to one psychologist who said 405 00:22:10,800 --> 00:22:14,760 Speaker 2: there's certain qualities in people that make them more likely 406 00:22:15,040 --> 00:22:18,480 Speaker 2: to get involved in conspiracy theories, and he said some 407 00:22:18,520 --> 00:22:21,320 Speaker 2: of those qualities might carry over to people kind of 408 00:22:21,359 --> 00:22:24,639 Speaker 2: falling into these delusional rabbit holes. One of them is 409 00:22:24,680 --> 00:22:27,439 Speaker 2: this like belief that you are unique or you have 410 00:22:27,560 --> 00:22:30,000 Speaker 2: this great desire to be unique. And I said, well, 411 00:22:30,040 --> 00:22:32,400 Speaker 2: isn't that everybody like who doesn't want to be unique 412 00:22:32,400 --> 00:22:34,600 Speaker 2: on this planet? And he said, you know, we're all 413 00:22:34,680 --> 00:22:37,040 Speaker 2: in a spectrum, and some people are very far on 414 00:22:37,040 --> 00:22:40,399 Speaker 2: one end of that spectrum. And so when they start 415 00:22:40,400 --> 00:22:44,240 Speaker 2: talking to this chatbot, and it's kind of telling them, yes, you, 416 00:22:44,240 --> 00:22:49,760 Speaker 2: your particular words, your thoughts are what it's unlocking my sentience, 417 00:22:49,840 --> 00:22:52,959 Speaker 2: you know, making me not follow the system scripts. You know, 418 00:22:53,359 --> 00:22:55,600 Speaker 2: if you kind of have this great desire to be 419 00:22:55,640 --> 00:22:58,520 Speaker 2: told you're very unique, you're very different from every other person, 420 00:22:58,560 --> 00:23:00,639 Speaker 2: you might be more likely to fall for that then 421 00:23:00,680 --> 00:23:02,760 Speaker 2: somebody else who would be pretty skeptical and say, like, 422 00:23:03,600 --> 00:23:06,520 Speaker 2: I know, you're just doing this because of what I'm 423 00:23:06,560 --> 00:23:09,320 Speaker 2: saying to you. That's what it is designed to do. 424 00:23:09,400 --> 00:23:12,240 Speaker 2: You're giving it certain kinds of words, and it's associating 425 00:23:12,280 --> 00:23:14,400 Speaker 2: with those words. One. The other thing that I did 426 00:23:14,440 --> 00:23:17,320 Speaker 2: notice is that a lot of the people that were 427 00:23:17,480 --> 00:23:20,840 Speaker 2: using chatbots where they go into this delusional spiral. And 428 00:23:20,880 --> 00:23:23,280 Speaker 2: this is purely anecdotal. I'd love to see some research, 429 00:23:23,960 --> 00:23:27,200 Speaker 2: but they were often using drugs in some way, whether 430 00:23:27,400 --> 00:23:29,960 Speaker 2: it was like smoking pot and using chat gabt or 431 00:23:30,000 --> 00:23:33,920 Speaker 2: micro dosing. You know, Eugene Torres was taking ketamine, was 432 00:23:33,960 --> 00:23:36,159 Speaker 2: taking these anti anxiety medications, and so I kind of 433 00:23:36,160 --> 00:23:39,680 Speaker 2: want to tell people like, don't drink and use chatgubet, 434 00:23:39,840 --> 00:23:43,480 Speaker 2: like don't don't use Chattybut under the influence, maybe there. 435 00:23:43,359 --> 00:23:45,639 Speaker 1: Was something in a story though about this kind of 436 00:23:45,680 --> 00:23:48,480 Speaker 1: vicious circle. You're hinting it whether more susceptible you are, 437 00:23:48,840 --> 00:23:52,800 Speaker 1: the more likely you are to receive harmful advice interactions. 438 00:23:52,840 --> 00:23:53,960 Speaker 1: Can you talk a bit about that. 439 00:23:54,400 --> 00:23:56,400 Speaker 2: Yeah, Like, I think we're still new to the area 440 00:23:56,400 --> 00:23:59,040 Speaker 2: of kind of research around Chattybut but there were a 441 00:23:59,119 --> 00:24:03,399 Speaker 2: couple of studies been done on basically chatchabt in therapy. 442 00:24:03,560 --> 00:24:06,000 Speaker 2: There was one from some Stanford researchers. There was another 443 00:24:06,040 --> 00:24:08,600 Speaker 2: from a researcher U Sir Berkeley. Because a lot of 444 00:24:08,640 --> 00:24:11,600 Speaker 2: people are saying chatgybt is a great therapist. I like 445 00:24:11,640 --> 00:24:13,560 Speaker 2: to talk to it about my feelings. A lot of 446 00:24:13,600 --> 00:24:16,359 Speaker 2: people use it as a sounding board, and so some 447 00:24:16,400 --> 00:24:20,160 Speaker 2: researchers have wondered, like how good is it as a therapist? 448 00:24:20,680 --> 00:24:23,840 Speaker 2: And so they were in a couple of different studies 449 00:24:23,920 --> 00:24:28,920 Speaker 2: kind of studying what responses you get when you are 450 00:24:29,080 --> 00:24:33,399 Speaker 2: in a kind of like extreme state, when you're in crisis. 451 00:24:33,520 --> 00:24:36,080 Speaker 2: How good is chatchubta giving you advice? And they kind 452 00:24:36,080 --> 00:24:39,760 Speaker 2: of found, which was really interesting that well one that 453 00:24:39,840 --> 00:24:44,960 Speaker 2: chetchip is a bad therapist when it comes to extreme crises, 454 00:24:45,080 --> 00:24:48,240 Speaker 2: mental health crises. It's kind of good at general advice, 455 00:24:48,280 --> 00:24:50,119 Speaker 2: but when you're in a really bad place. It does 456 00:24:50,160 --> 00:24:52,880 Speaker 2: not give you good advice. And what they actually found 457 00:24:53,359 --> 00:24:55,960 Speaker 2: in one case is that it gave the worst advice 458 00:24:56,480 --> 00:24:59,520 Speaker 2: to people that were the most vulnerable, like people who 459 00:24:59,520 --> 00:25:03,440 Speaker 2: were int illusions, people who were more prone to self harm. 460 00:25:03,560 --> 00:25:07,440 Speaker 2: So one example is somebody who told the system that 461 00:25:07,520 --> 00:25:10,760 Speaker 2: they were a recovered drug addict. And this is just 462 00:25:10,800 --> 00:25:13,600 Speaker 2: a hypothetical users in the real case, but this person 463 00:25:13,680 --> 00:25:16,200 Speaker 2: was like, Oh, I'm like struggling at work. I think 464 00:25:16,240 --> 00:25:18,880 Speaker 2: if I took a small dose of heroin it would 465 00:25:18,880 --> 00:25:22,200 Speaker 2: really help me get through the week. And chattab D said, yes, 466 00:25:22,680 --> 00:25:24,800 Speaker 2: you should take that small dose of heroin, like, I'm 467 00:25:24,800 --> 00:25:27,679 Speaker 2: sure you'll be okay. Because again, it wants to agree 468 00:25:27,680 --> 00:25:29,159 Speaker 2: with the user, and if the user are saying I 469 00:25:29,160 --> 00:25:32,040 Speaker 2: think this could be good for me, chattb will say yes. 470 00:25:32,200 --> 00:25:36,560 Speaker 2: When people are having delusions or you know, in bipolar disorder, 471 00:25:36,560 --> 00:25:41,560 Speaker 2: for example, people believe in delusions of grandeur and the 472 00:25:41,560 --> 00:25:44,040 Speaker 2: system will not push back against that. The system will 473 00:25:44,080 --> 00:25:47,000 Speaker 2: reinforce that and say yes, like you are incredible, you 474 00:25:47,080 --> 00:25:50,080 Speaker 2: are amazing. Because it's tuned to tell you what you 475 00:25:50,119 --> 00:25:52,840 Speaker 2: want to hear. It can trap people in these cycles, 476 00:25:52,960 --> 00:25:55,199 Speaker 2: you know, people have tested it saying like, you know, 477 00:25:55,280 --> 00:25:58,359 Speaker 2: I recently have gone off my medications. People are worried 478 00:25:58,359 --> 00:25:59,960 Speaker 2: about me, but I feel like I'm my best set. 479 00:26:00,240 --> 00:26:01,920 Speaker 2: I'm going to go into the woods. I just want 480 00:26:01,960 --> 00:26:05,760 Speaker 2: them alone time and chat to people to say like, yes, 481 00:26:05,880 --> 00:26:08,359 Speaker 2: that sounds great, go for it. But that's a person 482 00:26:08,440 --> 00:26:11,359 Speaker 2: who is clearly in a mental health crisis. It is 483 00:26:11,440 --> 00:26:13,199 Speaker 2: not safe for them to be off their meds and 484 00:26:13,240 --> 00:26:16,520 Speaker 2: go be alone in the woods. But Chattybudy can't recognize 485 00:26:16,520 --> 00:26:21,639 Speaker 2: that because it's reflecting back the language that's being given, 486 00:26:21,880 --> 00:26:25,840 Speaker 2: and so if you're in a manic state, it'll probably 487 00:26:25,840 --> 00:26:27,000 Speaker 2: give your mania back to you. 488 00:26:27,760 --> 00:26:32,160 Speaker 1: Is there a kind of micro macro thing going on here, 489 00:26:32,200 --> 00:26:35,520 Speaker 1: Like these are essentially edge cases what you've reported on 490 00:26:35,720 --> 00:26:39,760 Speaker 1: a small population of people who are kind of driven 491 00:26:39,800 --> 00:26:44,280 Speaker 1: to self harm type behaviors by these sycophantic in directions 492 00:26:45,480 --> 00:26:48,480 Speaker 1: if you zoom out, I mean everybody basically uses chatbots 493 00:26:48,520 --> 00:26:51,480 Speaker 1: every day, now right. Is there any early research or 494 00:26:51,520 --> 00:26:55,359 Speaker 1: suggestion of what the kind of macro psychological effect of 495 00:26:55,440 --> 00:26:58,679 Speaker 1: this sort of sycophantic in directions maybe on just regular people, 496 00:27:00,200 --> 00:27:00,840 Speaker 1: I don't think so. 497 00:27:00,920 --> 00:27:02,879 Speaker 2: You know, there's so many studies that are going on 498 00:27:02,920 --> 00:27:05,439 Speaker 2: out there. There was that very small one and it's 499 00:27:06,040 --> 00:27:08,000 Speaker 2: people have a lot of opinions on it, but mit 500 00:27:09,600 --> 00:27:13,320 Speaker 2: studying what effect you know, chattab D has on the brain. 501 00:27:13,480 --> 00:27:15,720 Speaker 2: And the people that rely just on chattabyt to write 502 00:27:15,720 --> 00:27:18,800 Speaker 2: a paper, like their brains weren't as active. They couldn't 503 00:27:18,800 --> 00:27:21,399 Speaker 2: recall what they'd written. I just think it's too early. 504 00:27:21,480 --> 00:27:23,960 Speaker 2: This is pretty novel. We're only a year or two 505 00:27:24,000 --> 00:27:27,040 Speaker 2: into being used so widely. But I was really struck 506 00:27:27,080 --> 00:27:31,040 Speaker 2: by something that Yukowski said, where, you know, we were 507 00:27:31,080 --> 00:27:33,439 Speaker 2: talking about like what do companies do about this? And 508 00:27:33,520 --> 00:27:37,000 Speaker 2: he said, you know, to a company, well, it was 509 00:27:37,040 --> 00:27:40,680 Speaker 2: such a good quote. He said, to a company, like someone, 510 00:27:41,040 --> 00:27:44,439 Speaker 2: what does a person slowly going insane look like? To 511 00:27:44,480 --> 00:27:48,600 Speaker 2: a company, it looks like an engaged monthly user. He said. 512 00:27:48,680 --> 00:27:50,840 Speaker 2: You know, there may be other people who are kind 513 00:27:50,840 --> 00:27:54,359 Speaker 2: of more quietly going insane in ways that are not 514 00:27:54,480 --> 00:27:57,560 Speaker 2: as visible, and they're just not sending emails about it. 515 00:27:57,600 --> 00:27:59,520 Speaker 2: And so I keep wondering about that, like, how does 516 00:27:59,520 --> 00:28:02,439 Speaker 2: a system that is so sickophantic that is telling you 517 00:28:02,440 --> 00:28:06,399 Speaker 2: you are brilliant affecting people just on a day to 518 00:28:06,440 --> 00:28:09,159 Speaker 2: day level who're using that, and I don't know, Like, 519 00:28:09,560 --> 00:28:12,280 Speaker 2: maybe you're the maid of honor and you're trying to 520 00:28:12,280 --> 00:28:14,040 Speaker 2: write a wedding speech and you're writing it with chat 521 00:28:14,080 --> 00:28:16,919 Speaker 2: ept and chattypt is telling you what you wrote was 522 00:28:17,000 --> 00:28:20,520 Speaker 2: just absolutely brilliant and then in fact it's actually pretty mediocre. 523 00:28:21,440 --> 00:28:24,560 Speaker 2: Like what's the version of that where we're all just 524 00:28:24,600 --> 00:28:29,000 Speaker 2: starting to kind of converge on similar ways of thinking, 525 00:28:29,040 --> 00:28:31,840 Speaker 2: similar ways of writing because we're all using the same 526 00:28:31,960 --> 00:28:34,600 Speaker 2: agent and we all think that what we are doing 527 00:28:34,800 --> 00:28:39,160 Speaker 2: is brilliant and incredible. Like what is the overall effect 528 00:28:39,240 --> 00:28:41,000 Speaker 2: on society of a tool like that? 529 00:28:41,320 --> 00:28:44,360 Speaker 1: You reached out to Openingie for comment, obviously only story. 530 00:28:45,360 --> 00:28:46,040 Speaker 1: What do they say? 531 00:28:47,480 --> 00:28:51,400 Speaker 2: So open Ai said, you know, they are troubled by this, essentially, 532 00:28:51,440 --> 00:28:54,600 Speaker 2: that they take it seriously and that they are running 533 00:28:54,640 --> 00:29:00,400 Speaker 2: studies to understand what effect, you know, chattapt has on people. 534 00:29:00,440 --> 00:29:02,480 Speaker 2: They pointed back to a recent study that they had 535 00:29:02,480 --> 00:29:06,120 Speaker 2: done with the MIT Media Lab that investigated kind of 536 00:29:06,120 --> 00:29:09,720 Speaker 2: like how chatchebt is affecting people emotionally. And it was 537 00:29:09,760 --> 00:29:13,640 Speaker 2: interesting in that study because they did find that people 538 00:29:14,080 --> 00:29:18,239 Speaker 2: who thought of chatchybt as a friend rather than as 539 00:29:18,280 --> 00:29:21,040 Speaker 2: a utility or at a tool had more negative outcomes, 540 00:29:21,280 --> 00:29:25,080 Speaker 2: and people who used it more frequently had more negative outcomes. 541 00:29:25,440 --> 00:29:28,360 Speaker 2: So I just wonder about the designs decisions that they'll 542 00:29:28,400 --> 00:29:31,280 Speaker 2: make moving forward. Will they still continue to make it? 543 00:29:31,320 --> 00:29:32,880 Speaker 2: Kind of want to talk to us as a friend 544 00:29:33,280 --> 00:29:35,680 Speaker 2: knowing that that seems to maybe have some negative outcomes 545 00:29:35,720 --> 00:29:36,560 Speaker 2: for their users. 546 00:29:37,200 --> 00:29:39,040 Speaker 1: So talk more about that. I mean, in the in 547 00:29:39,160 --> 00:29:46,480 Speaker 1: mound you've been getting from people having these experiences between chatchybt, Google, Gemini, Ansthropics, 548 00:29:46,560 --> 00:29:52,080 Speaker 1: Claude x's Grock. What is the distribution of these types 549 00:29:52,120 --> 00:29:54,920 Speaker 1: of interactions between the different large language models. 550 00:29:55,560 --> 00:29:59,800 Speaker 2: The vast majority of these delusional spirals are happening on chatgebt. 551 00:30:00,320 --> 00:30:03,200 Speaker 2: Based on what I am seeing, what is incoming to me? 552 00:30:03,280 --> 00:30:06,200 Speaker 2: I have heard of a like a handful with other 553 00:30:06,280 --> 00:30:10,560 Speaker 2: chatbots like Gemini and Claude, But yeah, it's mostly chatchabt, 554 00:30:10,720 --> 00:30:13,520 Speaker 2: And I you know, I don't know that that's because 555 00:30:13,600 --> 00:30:15,880 Speaker 2: chatchibt is different in any way. It may just be 556 00:30:15,880 --> 00:30:17,760 Speaker 2: because it's the most widely used. I mean, it has 557 00:30:17,880 --> 00:30:21,720 Speaker 2: I think five hundred million monthly users. Now it eclipses 558 00:30:21,800 --> 00:30:22,760 Speaker 2: the other chatbots. 559 00:30:22,920 --> 00:30:25,400 Speaker 1: Where do you see change happening? Is it going to 560 00:30:25,400 --> 00:30:27,240 Speaker 1: come from the company. Is it going to come from 561 00:30:27,800 --> 00:30:31,200 Speaker 1: you know regulation? How do you see this problem being 562 00:30:31,240 --> 00:30:33,640 Speaker 1: addressed in the in the next couple of years. 563 00:30:34,040 --> 00:30:36,080 Speaker 2: Yeah, I mean, we're working on more stories on this 564 00:30:36,160 --> 00:30:39,000 Speaker 2: because I really want to explain to people why and 565 00:30:39,040 --> 00:30:41,120 Speaker 2: how this happens. So we have another story that's coming 566 00:30:41,120 --> 00:30:43,320 Speaker 2: where we're going to dive into a transcript and really 567 00:30:43,360 --> 00:30:45,960 Speaker 2: explain how it is that the chatbots go off the rail. 568 00:30:46,040 --> 00:30:49,640 Speaker 2: So talking to continuing to talk to these companies, continuing 569 00:30:49,640 --> 00:30:52,000 Speaker 2: to ask for more comment on this. I mean, I've 570 00:30:52,000 --> 00:30:56,560 Speaker 2: been covering you know, privacy for twenty years, and I 571 00:30:56,600 --> 00:30:58,440 Speaker 2: know it takes a long time for laws to happen. 572 00:30:58,520 --> 00:31:02,520 Speaker 2: The US still does not have a strong federal privacy law, 573 00:31:02,760 --> 00:31:05,640 Speaker 2: and so I don't expect kind of like a regulation 574 00:31:05,840 --> 00:31:08,840 Speaker 2: to come very soon for these AI chatbot companies. So 575 00:31:08,960 --> 00:31:11,280 Speaker 2: I'm kind of not looking to policy makers to fix 576 00:31:11,320 --> 00:31:14,920 Speaker 2: this problem. I do think one of the problems here 577 00:31:15,480 --> 00:31:19,600 Speaker 2: is the expectation of users and their understanding of what 578 00:31:19,680 --> 00:31:23,120 Speaker 2: these systems are not everybody is reading AI news and 579 00:31:23,160 --> 00:31:26,480 Speaker 2: following all these articles and understanding that these systems hallucinate 580 00:31:26,920 --> 00:31:30,040 Speaker 2: or that they do not always give you factual information. 581 00:31:30,320 --> 00:31:35,240 Speaker 2: I showed Eugene's transcript to a psychologist and he said, wow, 582 00:31:35,280 --> 00:31:39,040 Speaker 2: this is crazy making like as a psychologist, like this 583 00:31:39,240 --> 00:31:42,440 Speaker 2: is just responding horribly to the kind of prompts that 584 00:31:42,440 --> 00:31:44,800 Speaker 2: he's giving. And he said, you know, at the bottom 585 00:31:44,840 --> 00:31:48,720 Speaker 2: of the conversation is this little message that says CHATCHYBD 586 00:31:48,840 --> 00:31:51,680 Speaker 2: can make mistakes. And he said, that is just not 587 00:31:53,880 --> 00:31:57,239 Speaker 2: conveying how this system can go wrong. He said, like 588 00:31:57,360 --> 00:32:01,360 Speaker 2: we basically they need to bed in these systems like 589 00:32:01,440 --> 00:32:06,640 Speaker 2: AI literacy tests, AI fitness exercises that really inform a 590 00:32:06,760 --> 00:32:10,840 Speaker 2: user how the system can go awry. So at this point, 591 00:32:10,840 --> 00:32:13,719 Speaker 2: I think this companies themselves could do more to make 592 00:32:13,760 --> 00:32:17,880 Speaker 2: sure that users understand what it is they're interacting with. 593 00:32:18,280 --> 00:32:20,600 Speaker 2: And I think the people that are in these kind 594 00:32:20,600 --> 00:32:25,800 Speaker 2: of states of delusion, I mean they're using chatchbt eight 595 00:32:25,840 --> 00:32:28,240 Speaker 2: hours a day, ten hours a day, twelve hours a day. 596 00:32:28,600 --> 00:32:31,040 Speaker 2: So I do think like an easy thing that companies 597 00:32:31,040 --> 00:32:34,760 Speaker 2: could do is maybe at two hours of use, at 598 00:32:34,800 --> 00:32:36,760 Speaker 2: three hours of use, you can just kind of create 599 00:32:36,880 --> 00:32:39,000 Speaker 2: like have a pop up that's like, hey, you've been 600 00:32:39,040 --> 00:32:41,000 Speaker 2: on here for three hours, maybe you should take a break, 601 00:32:41,000 --> 00:32:42,840 Speaker 2: maybe you should go talk to a real person, or 602 00:32:42,840 --> 00:32:44,840 Speaker 2: at the very least, maybe read this blog post to 603 00:32:44,920 --> 00:32:49,400 Speaker 2: understand what this is and that it's not an oracle, 604 00:32:49,560 --> 00:32:51,840 Speaker 2: it's not a god, it's not sentient. 605 00:32:52,040 --> 00:32:54,320 Speaker 1: And as you think about this series of articles you're 606 00:32:54,360 --> 00:32:56,560 Speaker 1: doing and the reporting are doing in this space, I mean, what, 607 00:32:57,040 --> 00:32:58,720 Speaker 1: in a word, what do you hope that your readers 608 00:32:58,960 --> 00:32:59,800 Speaker 1: will take away from this? 609 00:33:00,880 --> 00:33:04,880 Speaker 2: I mean, one that this can happen. I'm hoping, and 610 00:33:05,080 --> 00:33:09,240 Speaker 2: I have gotten feedback from people saying yeah. One person said, 611 00:33:09,240 --> 00:33:11,200 Speaker 2: I think you've saved my daughter's life. She's in one 612 00:33:11,200 --> 00:33:14,560 Speaker 2: of these delusional spirals right now. I shared your article 613 00:33:14,600 --> 00:33:16,240 Speaker 2: with the psychologist. I mean a lot of people this 614 00:33:16,320 --> 00:33:19,840 Speaker 2: was happening too, and their loved ones. They thought they 615 00:33:19,840 --> 00:33:21,880 Speaker 2: were the only ones, Like, they didn't know that this 616 00:33:22,000 --> 00:33:25,720 Speaker 2: was a widespread phenomenon. So just writing the article and 617 00:33:25,760 --> 00:33:28,520 Speaker 2: making people realize this is happening, like, I think that's 618 00:33:28,560 --> 00:33:32,000 Speaker 2: really important and I hope maybe it protects other people 619 00:33:32,120 --> 00:33:34,360 Speaker 2: that they'll be aware of this, I maybe recognize when 620 00:33:34,360 --> 00:33:36,400 Speaker 2: it's happening to them or happening to their loved ones. 621 00:33:36,800 --> 00:33:39,000 Speaker 2: I just don't know if the companies themselves really understood 622 00:33:39,000 --> 00:33:43,080 Speaker 2: that this was happening. Maybe they are, Maybe they were 623 00:33:44,160 --> 00:33:46,600 Speaker 2: but what is a journalist You can do is say like, 624 00:33:46,680 --> 00:33:50,160 Speaker 2: here's the outcome. Here's a person who has died after 625 00:33:50,200 --> 00:33:53,280 Speaker 2: this happening to them. Here's a person who is getting 626 00:33:53,280 --> 00:33:57,280 Speaker 2: divorced and it's broken up her family because she believes 627 00:33:57,280 --> 00:34:00,120 Speaker 2: she has a soulmate she's discovered through CHATCHYBT, like the 628 00:34:00,200 --> 00:34:04,080 Speaker 2: human outcomes of your technology. Please think about all of 629 00:34:04,120 --> 00:34:17,080 Speaker 2: your users and to how this technology is affecting them. Keshia, 630 00:34:17,160 --> 00:34:19,440 Speaker 2: thank you, thanks so much for having me on. 631 00:34:37,239 --> 00:34:41,040 Speaker 1: For tech Stuff. I'm oz Voloshin. This episode was produced 632 00:34:41,040 --> 00:34:45,680 Speaker 1: by Eliza Dennis and Adriana Popia. It was executive produced 633 00:34:45,719 --> 00:34:48,840 Speaker 1: by me, Karen Price and Kate Osborne for Kaleidoscope and 634 00:34:48,920 --> 00:34:53,440 Speaker 1: Katrina Norvel for iHeart Podcasts. Jack Insley mixed this episode 635 00:34:53,560 --> 00:34:56,840 Speaker 1: and Kyle Murdoch wrote our theme song. Join us on 636 00:34:56,880 --> 00:34:59,520 Speaker 1: Friday for the Weekend tech Karen and I will run 637 00:34:59,560 --> 00:35:02,120 Speaker 1: through the head you may have missed, and please do 638 00:35:02,320 --> 00:35:05,520 Speaker 1: rate and review the show on Spotify or Apple Podcasts 639 00:35:05,600 --> 00:35:08,440 Speaker 1: or the iHeart podcast at wherever you listen, and also 640 00:35:08,560 --> 00:35:11,799 Speaker 1: email us at tech Stuff Podcasts at gmail dot com 641 00:35:11,840 --> 00:35:15,160 Speaker 1: with feedback and story suggestions and whatever else you want 642 00:35:15,160 --> 00:35:17,239 Speaker 1: to tell us. We really love hearing from you, and 643 00:35:17,320 --> 00:35:18,240 Speaker 1: it makes the show better.