Stony Brook study shows AI can measure public's mental health through social media

A new study shows artificial intelligence can measure how much depression and anxiety is in a community by tracking the language used on social media.

The study, done by Stony Brook University researchers working with other academic institutions, found the accuracy of the AI assessment was comparable to standard population surveys, done through phone polling. But artificial intelligence can produce more precise and timely information at a lower cost, the study found.

“Traditionally, if you want to do this type of work, you need to call 50,000 people and spend $1 million to get some notion” of people's mental health, said Andrew Schwartz, an associate professor of computer science and psychology at Stony Brook and senior author of the study.

Its findings appear in the May 2 edition of “Nature Digital Medicine” in the article “Robust language-based mental health assessments in time and space through social media.” The lead study author, Siddharth Mangalik, said AI was able to pinpoint spikes in anxiety and depression with major crises and social events, like George Floyd's murder by a Minneapolis police officer and subsequent nationwide protests and the onset of the pandemic. 

Schwartz said using artificial intelligence to examine language people used in public, preexisting social media posts enabled researchers to get information down to the county level on a weekly basis. He said the AI analysis can also be done more frequently than polling.

The AI program measured language with markers for depression and anxiety in 1 billion tweets for more than 2 million geolocated users from 2019 to 2020 in 1,418 counties — about half of U.S. counties — where researchers said 90% of the population resides.

The researchers compared the AI assessment method with polling done by the Gallup organization during the same period. Schwartz said the AI measures were similar to Gallup's.

Schwartz said in phone surveys people's answers are usually converted to numbers. He said a respondent might be asked to answer, on a scale of 1 to 5, the question: "To what extent you felt sad in the past week?”

“Ultimately what's recorded is a number … They could be upset because they had a flat tire that day. They might be upset about that and say they are sad about that, and that would be treated the same as someone who says they are sad because a loved one died.”

Those two examples shouldn't carry the same significance, he suggested.

With the AI tool, “people can answer in their natural everyday words, rather than being forced to say on a scale of 1 to 5, how sad are you?” Schwartz said.

Mangalik, a Stony Brook doctoral student in computer science and lead author of the study, said through the AI program's assessments of people's language usage, “We're observing the behavior of people rather than asking people to self-report their feelings of sadness or worry.”

The benefit to the public, Mangalik said, was that policymakers, public health officials and researchers could use the AI program to “pinpoint where in the state or community and pinpoint the time” when people express mental health concerns.

Mangalik said the program flags “words like worry, stress, alone.” Seeing words like those “over and over,” he said, are a “strong indicator” of depression or anxiety.

On Long Island, Mangalik said the data show that depression spiked both in Nassau and Suffolk counties in May and June 2020, around the time Floyd was killed, and other spikes around the start of the pandemic earlier in 2020.

“For both Suffolk and Nassau counties [researchers] observed large increases in community depression scores in 2020,” Mangalik added in an email. “Nassau County in particular saw over a tripling in their scaled depression scores from February 2020 (0.191) to May 2020 (0.615), while Suffolk County over doubled in the same time period (0.287 to 0.580). By August 2020 … both communities largely recovered to their previous levels.”

Schwartz said the researchers, which include academics at Stanford University and the University of Pennsylvania, are hoping to establish a website by October “so that anyone can explore how their communities' anxiety and depression has changed recently.”

Janine Logan, vice president of communications and population health for the Nassau-Suffolk Hospital Council and director of the Long Island Health Collaborative, said in an email: “this research … is another helpful resource for public health leaders and those who treat mental illness.”

Gov. Kathy Hochul said in a statement  Thursday she was seeking to increase AI research in the state, announcing several AI initiatives at State University of New York campuses such as Stony Brook that will be for the “public good.” She said $275 million was in the 2025 budget for Empire AI, to scale up AI research and scholarship and internship opportunities for students at SUNY, and create departments and centers of AI across select SUNY campuses.

She added that a new chatbot program, tailored to coursework, research and student projects, “will make SUNY the largest AI Large Language Model-enabled education system in the world.”

By Olivia Winslow

olivia.winslow@newsday.com@newsollie