Stony Brook study shows AI can measure public's mental health through social media
The main entrance to the Stony Brook University West Campus. Stony Brook researchers conducted a study on how artificial intelligence can measure the public's mood through monitoring social media. Credit: Newsday/John Paraskevas
A new study shows artificial intelligence can measure how much depression and anxiety is in a community by tracking the language used on social media.
The study, done by Stony Brook University researchers working with other academic institutions, found the accuracy of the AI assessment was comparable to standard population surveys, done through phone polling. But artificial intelligence can produce more precise and timely information at a lower cost, the study found.
Only 25¢ for 5 months
A new study shows artificial intelligence can measure how much depression and anxiety is in a community by tracking the language used on social media.
The study, done by Stony Brook University researchers working with other academic institutions, found the accuracy of the AI assessment was comparable to standard population surveys, done through phone polling. But artificial intelligence can produce more precise and timely information at a lower cost, the study found.
“Traditionally, if you want to do this type of work, you need to call 50,000 people and spend $1 million to get some notion” of people's mental health, said Andrew Schwartz, an associate professor of computer science and psychology at Stony Brook and senior author of the study.
Its findings appear in the May 2 edition of “Nature Digital Medicine” in the article “Robust language-based mental health assessments in time and space through social media.” The lead study author, Siddharth Mangalik, said AI was able to pinpoint spikes in anxiety and depression with major crises and social events, like George Floyd's murder by a Minneapolis police officer and subsequent nationwide protests and the onset of the pandemic.
WHAT TO KNOW
- A study by Stony Brook researchers, who partnered with other institutions, shows artificial intelligence can measure the mental health of a community through monitoring social media
- The study authors say AI measured depression and anxiety on Long Island in 2020 and showed spikes during the pandemic and the police killing of George Floyd.
- The AI assessment could be used to give policymakers a tool for assessing the public's mood at any given time in a way comparable to standard population surveys, the authors say.
Schwartz said using artificial intelligence to examine language people used in public, preexisting social media posts enabled researchers to get information down to the county level on a weekly basis. He said the AI analysis can also be done more frequently than polling.
The AI program measured language with markers for depression and anxiety in 1 billion tweets for more than 2 million geolocated users from 2019 to 2020 in 1,418 counties — about half of U.S. counties — where researchers said 90% of the population resides.
The researchers compared the AI assessment method with polling done by the Gallup organization during the same period. Schwartz said the AI measures were similar to Gallup's.
Schwartz said in phone surveys people's answers are usually converted to numbers. He said a respondent might be asked to answer, on a scale of 1 to 5, the question: "To what extent you felt sad in the past week?”
“Ultimately what's recorded is a number … They could be upset because they had a flat tire that day. They might be upset about that and say they are sad about that, and that would be treated the same as someone who says they are sad because a loved one died.”
Those two examples shouldn't carry the same significance, he suggested.
With the AI tool, “people can answer in their natural everyday words, rather than being forced to say on a scale of 1 to 5, how sad are you?” Schwartz said.
Mangalik, a Stony Brook doctoral student in computer science and lead author of the study, said through the AI program's assessments of people's language usage, “We're observing the behavior of people rather than asking people to self-report their feelings of sadness or worry.”
The benefit to the public, Mangalik said, was that policymakers, public health officials and researchers could use the AI program to “pinpoint where in the state or community and pinpoint the time” when people express mental health concerns.
Mangalik said the program flags “words like worry, stress, alone.” Seeing words like those “over and over,” he said, are a “strong indicator” of depression or anxiety.
On Long Island, Mangalik said the data show that depression spiked both in Nassau and Suffolk counties in May and June 2020, around the time Floyd was killed, and other spikes around the start of the pandemic earlier in 2020.
“For both Suffolk and Nassau counties [researchers] observed large increases in community depression scores in 2020,” Mangalik added in an email. “Nassau County in particular saw over a tripling in their scaled depression scores from February 2020 (0.191) to May 2020 (0.615), while Suffolk County over doubled in the same time period (0.287 to 0.580). By August 2020 … both communities largely recovered to their previous levels.”
Schwartz said the researchers, which include academics at Stanford University and the University of Pennsylvania, are hoping to establish a website by October “so that anyone can explore how their communities' anxiety and depression has changed recently.”
Janine Logan, vice president of communications and population health for the Nassau-Suffolk Hospital Council and director of the Long Island Health Collaborative, said in an email: “this research … is another helpful resource for public health leaders and those who treat mental illness.”
Gov. Kathy Hochul said in a statement Thursday she was seeking to increase AI research in the state, announcing several AI initiatives at State University of New York campuses such as Stony Brook that will be for the “public good.” She said $275 million was in the 2025 budget for Empire AI, to scale up AI research and scholarship and internship opportunities for students at SUNY, and create departments and centers of AI across select SUNY campuses.
She added that a new chatbot program, tailored to coursework, research and student projects, “will make SUNY the largest AI Large Language Model-enabled education system in the world.”
'I'm going to try to avoid it' A trip to the emergency room in a Long Island hospital now averages nearly 4 hours, data shows. NewsdayTV's Virginia Huie reports.