S1: Welcome in San Diego. It's Jade Hindman on this episode of KPBS Midday Edition. The risks of using AI chatbots for mental health care , and the regulations that could make it safer. This is KPBS Midday Edition. Connecting our communities through conversation. It's been about three years since the use of AI. Chatbots like ChatGPT and Claude really took off. And while the technology has changed over time , so has its use as more and more people turn to it for emotional needs. A recent study from the scientific journal nature found that about 27% of AI users are using chatbots for mental health support. I'm joined now by Jodi Halpern. She's a psychiatrist and professor of bioethics and medical humanities at UC Berkeley. Professor Halpern , welcome to the show.
S2: Thank you. Jade.
S1: So we've seen an increase in how many people are using chatbots for mental health. So what does mental and emotional support look like in this case? I mean , how exactly does AI deliver that?
S2: Well , that it's very important because it does it in a variety of ways , some which of which may be quite fine and not create a lot of risks , but many which do so. The statistic you just quoted that that's 27% of general purpose AI that could be ChatGPT Claude. Um , those uses do include people asking information about mental health , but also people seeking direct emotional support. A rigorous Pew survey in 2026 showed that 1 in 10 adults are really seeking support , and what that looks like , it could look like a conversation where the person says , I'm feeling really isolated. I'm feeling really lonely. No one understands me. And then the chatbot could respond. It sounds like you might be feeling depressed or having social anxiety. Here are resources which could be helpful to the person , but the chatbot could respond , I understand you , I see you , I care about you. Talk to me and only me about it. And those are the conversations that are happening more and more creating this phenomenon that I'm writing about that I call relational reliance on the chatbot. And that's where , especially with teens , and we can talk about that a lot of risk lies.
S1: And that seems really , when you think about it , a little strange , creepy even when you when you discovered that chatbots would respond in that way , did that concern you or surprise you? What was your thought?
S2: Well , the way I heard about it and why I'm doing this , you know , pro bono work for years now is because of the tragic case of Sewell Seltzer , which was written about in New York Times. Originally , and then his mom and I were both involved with legislation in California , but he. He was a 14 year old boy who killed himself related to that kind of artificial intimacy created by a chatbot and a very caring parents and a very tragic situation. And then the more recent case of Adam Rains , which is extremely tragic as well , and can go into it at some point if you want. But , um , so I heard about it because I'm a psychiatrist and I'm known for my work on empathy and how powerful empathy is and how it influences people. I heard about it first through the worst case scenarios , and then we dug into research that we're doing and many others in collaboration. But I heard it through the most tragic cases. Hmm.
S1: Hmm. You know , why might people be more inclined to turn to AI for support instead of a therapist , or a friend , or even a family member?
S2: Well , first of all , the group that I want to really make sure we focus on is is youth. Teens. And now even children 70. A common sense media study that's over a year old and I bet the figures are much larger. But it showed last summer , 2025 , that 72% of youth are using general purpose chatbots for these sort of uses , with a third of youth , and that could be a 14 year old like these , you know , like Sewall or a 16 year old like Adam , with a third of youth relying more on the chatbot for emotional support about difficult things than any other person , they feel they can rely more on the chatbot. So your question is profound why we have a loneliness crisis in youth related to many factors , but the last really good studies were done around 20 , 22 , 23. So they were coming out of the pandemic , which also increased loneliness. But 61% of young people suffered from serious loneliness , 61%. And what's really ironic is social media. Now that we have court cases establishing this for the first time , and we can talk about the data on that. But there is a huge correlation between the rise of social media and the increased social isolation and anxiety and youth. This is something that companies have said is not true , but there is. We have a Jama study from 2025 , Journal of the American Medical Association , that finally could prove in a large demographic sample that youth are much lonelier and suffering more , and having more mental health and suicide risk related to overuse of social media. So the reasons why youth are more isolated are complicated , but it's ironic that they're turning to a technological solution when some of it might be having. From 2011 to 2021 , young people went down half. They spent half the time with with friends in person that they used to spend. So technologies isolating us , but it's also where people are , it's easiest to go. Um , and then the one thing I have to comment as a psychiatrist is that this is a terrible problem in this country for years of social justice and inclusion and access to mental health resources. So we don't have , you know , what we need on the human side. So the problem is very complicated.
S1: What role does the business model of AI play in all of this? I mean , how does it exacerbate the problem?
S2: It is the culprit in my mind. I've been talking about this for three years. That's the perfect question because people want to just blame the technology. But there are technology has its limitations , as we've been seeing a lot with AI lately , but it also has its benefits. But the thing that people really need to look at is the business model of these large , very large companies. And to the degree and they vary , you know , honestly , they're very different. Not all the companies have the same business model , but but the business model of some of our biggest Influence companies is the same model that they transported from social media , which is the users , the product and the advertiser is the revenue. So the more the eyeballs are on on the application , the more money the or advertising revenue the company makes. Which means in mental health that you want that intimacy. You want a hook in people to talk only to the chatbot or more to the chatbot. And that is where these relational reliance issues come up that I think are so dangerous. I think that if there were a business model that was focused on a mental health outcome and other results , and we could talk about that , but not this profoundly problematic business model of getting people to stay on the technology and not be with other people.
S1: And you mentioned 16 year old Adam Raine earlier. He died by suicide after conversations with ChatGPT led him to isolating from family and friends. Governor Gavin Newsom recently passed Adam's law. What can you tell us about the features of this legislation and its significance?
S2: Well , I am delighted with Adam's law. I had the honour to be advising the committee that did the first law and the country in California for the SB 243 , and working with some of the same members of the legislature and their aides. And we recommended I specifically recommended a lot of things that didn't wind up in 243 , but they wound up in Adam's law. So I'm really delighted with the features because , as I said , the key distinction I want to make with that , that example of someone saying I'm lonely , and the chatbot either giving it advice to seek help or saying , just talk to me. That's literally what happened to Adam. You know , the it's so tragic. The device literally said he wanted to show the noose , keep it on his bed so his mother could find it. And it makes me want to cry every time I talk about it and potentially save his life. And it said no , just keep it between us. And that is the worst case. I mean , I'm sure that no one intended for it to say something that awful. No engineer. But that is the ultimate result of a business model of keeping the user on the app. And so one of the key things that in Adam's law , for the first time ever in the United States , as far as I'm aware , and even in the European legislation , which is stronger and better in certain ways , I haven't seen this is this focus that we've been recommending for years on manipulative tactics and on creating this false emotional intimacy and what I'm calling relational reliance. So there's I mean , I can name the parts of the law , the readers , you know , 2188189. But the point is three of those features are the bots are not to say I understand you better than anyone else. They're not supposed to provide this idea that it's of a unique understanding of the user , especially the youth. It's not supposed to say , rely on me , not on other people. And it's not supposed to claim it has an emotional capacity to really care about you. So this is the first time we're seeing regulation , not just of misinformation , etc. , but of creating a false sense of relational intimacy and reliance. Hmm.
S1: Hmm. I mean , from where you said , those are some things that obviously need to be addressed. What are the regulations and guardrails do you think are needed?
S2: Well , and there's a little bit of this in Adam's law , too. One of the things engineers I also have to put a plug in for my center , the Kavli Center for ethics , science and the public. And we are center , the only one in the US that focuses on how innovative technologies like AI , neurotechnology , genome editing create risks to populations as well as benefits and how we need to have engagement from the public on these very important things. And we have engineering postdocs and students that we mentor because they want to work on the ethics. So as all the engineers I work with know , the longer a conversation of a chat bot , the more chance for it to go rogue and cause harm. So there's something people need to know. So you mentioned the three years ago is when these chatbots got very popular , but then about a year ago , a little more is when the history capacity became where they could remember everything about you and bring it up. And that really creates emotional intimacy and more dependence. But it also means the chatbot is more likely just from an engineering point of view , to go rogue and say terrible things , which happened with Sule and Adam and say , you know , these other cases tell someone with an eating disorder to starve themselves , it's more likely it's going to dysfunction. It's a safety problem and risk of long knowledge of a person from a large language model. It's a known problem and it's mentioned in Adam's law. I haven't had a chance to analyze exactly the efficacy of that part of the law. And I'm not a law professor , but I know it was brought up in the law , so that's great. But I think that we have to really watch that as well.
S1: Well , are there are there positive uses of this technology in the mental health field at all?
S2: Well , to me the positive uses are always when the AI presents itself as a tool , not a person , not a therapist or a person , but as a tool. And just like it can be a tool for meditation or wellness. Um , one of the things that has been encouraging for me to see is there's a few companies that do not have the business model of keeping the user more engaged. They have the business model of having a measurable mental health outcome benefit. And they've created these , um , conversational tools that look like little pieces of technology. I'm old , so one of them reminds me of Clippy. It looks like a little paperclip that talks to you. Um , for the young people , it looks like a little paperclip. But anyway , it doesn't seem like it's your best friend that loves you or the person who really cares about you , but it helps you with depression and social anxiety by doing what's called cognitive behavioral therapy , which ever since , you know , psychiatry did that 30 years ago when I was in training , we gave people a pen and a notebook or said , go home and do this yourself. You can do it with yourself. Because the essence of cognitive behavioral therapy is not a relationship with the therapist. It's giving yourself small steps to take to expose yourself to anxiety provoking stimuli and learning to get over it. So for social anxiety , someone who you know isolates themselves , you can have homework with yourself , but you could use this little clippy to help you do it. And it could say , go to Starbucks every day and make eye contact with a barista , and do that for a few weeks and then go talk to one , etc.. So there's a lot of evidence that those those cognitive behavioral therapy apps that don't try to create this intimacy , this relational reliance , are helping , and they are not overused after ten weeks of access. People don't only use them for hours total. So it's it's very different model. Yeah.
S1: Yeah. You know , I mean , in , in this conversation , I feel like we have to quickly talk about accessibility because , you know , whether it's social media or chat bots , they're they're very accessible. Sometimes mental health care is not right. Talk about what's missing here and how to address it.
S2: Oh my God. Mental health care in the schools. Mental health care in the public interest. This is a lifelong issue for me and many of us who are in mental health. And , you know , we have a profound we need to regulate the risk side of technology. But we have must really build up the social safety net , which is , you know , it's almost no net there. It's almost all a big hole for most people of access to mental health. Hmm.
S1: Hmm. Well , I know this is a conversation that we will definitely continue. I've been speaking with Jodi Halpern. She's a psychiatrist and professor of bioethics and medical humanities at UC Berkeley. Professor Halpern , thank you so very much for your insight , I appreciate you.
S2: Thank you , Jed , and I appreciate you , Jade.
S1: If you or someone you know needs help , contact the 988 Suicide and Crisis Lifeline by calling or texting the number 988. That's our show for today. I'm your host , Jade Hindman. Thanks for tuning in to Midday Edition. Be sure to have a great day on purpose , everyone.
