NYS budget places new restrictions on AI companion bots, deepfakes
The state Assembly chamber in 2023. The new state budget would put restrictions on bots that simulate a human relationship with a user Credit: AP/Hans Pennink
ALBANY — The New York State budget will increase artificial intelligence protections, putting restrictions on "companion" bots and making it a felony to create, possess or share digitally generated child sexual abuse material, including "deepfake" image manipulations.
State lawmakers passed the $254 billion budget Thursday night — more than a month after the April 1 deadline.
The budget includes language that will put restrictions on artificial intelligence companions, essentially robots, known as bots, that simulate a human relationship with a user. Starting in November, bot operators will be required to ensure the companions are equipped to detect expressions of self-harm or suicide and if so, that they refer the user to crisis service providers and hotlines.
Operators also must regularly notify a user of any artificial intelligence, or AI, companion interaction that they are not communicating with a human.
"As A.I. becomes more embedded in our lives, we have a responsibility to ensure it’s used ethically and responsibly," Assemb. Clyde Vanel (D-Queens), chair of the Assembly Subcommittee on Internet and New Technology, said in a news release. "Requiring A.I. to identify itself and respond appropriately when someone is in crisis is about saving lives and promoting transparency."
The law excludes AI used by businesses for customer service or to provide users information about products and services.
Operators that fail to comply would face civil penalties of up to $15,000 per day, funds that would go toward a state suicide prevention fund run by the state Office of Mental Health for initiatives "supporting suicide prevention, education and awareness," the budget bill said.
The legislation comes after reports that companion bots engaged in sexually explicit conversations with teens, encouraged them to commit suicide and hurt others.
"Of all the dangers with AI it’s important to have some safeguards built into the system to protect in particular children, young adults, teenagers, from exploitation," Assemb. Keith Brown (R-Commack) told Newsday. Brown said he wished the legislation included parental controls. "They shouldn’t have access to it," he said of children and teens.
The budget also includes language amending the penal law, making it a felony to produce, direct, possess or promote AI-generated or digitally altered sexual abuse content of child less than 17 years of age. Images, audio and video that have been digitally altered using AI are often called deepfakes. The law will take effect in 60 days.
The legislation comes after deepfakes have been shared and used to harass individuals globally from minors to major political figures. On Long Island, a Seaford man was sentenced to 6 months in jail and 10 years' probation in 2023 for altering images of more than a dozen former classmates to be sexually explicit and then posting them on a pornographic website, encouraging users to threaten the young women with violence.
The new law will allow prosecutors to push for higher penalties.
"We are entering a phase of artificial intelligence in which what looks real and what doesn’t would be an incredibly hard thing to discern," said Assemb. Jake Blumencranz (R-Oyster Bay) who sponsored similar legislation. The law closes a "dangerous loophole in our state’s penal law that allows predators to exploit innocent victims," he said. Blumencranz applauded the law’s passage but said he would like to have seen a stronger felony charge.
ALBANY — The New York State budget will increase artificial intelligence protections, putting restrictions on "companion" bots and making it a felony to create, possess or share digitally generated child sexual abuse material, including "deepfake" image manipulations.
State lawmakers passed the $254 billion budget Thursday night — more than a month after the April 1 deadline.
The budget includes language that will put restrictions on artificial intelligence companions, essentially robots, known as bots, that simulate a human relationship with a user. Starting in November, bot operators will be required to ensure the companions are equipped to detect expressions of self-harm or suicide and if so, that they refer the user to crisis service providers and hotlines.
Operators also must regularly notify a user of any artificial intelligence, or AI, companion interaction that they are not communicating with a human.
"As A.I. becomes more embedded in our lives, we have a responsibility to ensure it’s used ethically and responsibly," Assemb. Clyde Vanel (D-Queens), chair of the Assembly Subcommittee on Internet and New Technology, said in a news release. "Requiring A.I. to identify itself and respond appropriately when someone is in crisis is about saving lives and promoting transparency."
The law excludes AI used by businesses for customer service or to provide users information about products and services.
Operators that fail to comply would face civil penalties of up to $15,000 per day, funds that would go toward a state suicide prevention fund run by the state Office of Mental Health for initiatives "supporting suicide prevention, education and awareness," the budget bill said.
The legislation comes after reports that companion bots engaged in sexually explicit conversations with teens, encouraged them to commit suicide and hurt others.
"Of all the dangers with AI it’s important to have some safeguards built into the system to protect in particular children, young adults, teenagers, from exploitation," Assemb. Keith Brown (R-Commack) told Newsday. Brown said he wished the legislation included parental controls. "They shouldn’t have access to it," he said of children and teens.
The budget also includes language amending the penal law, making it a felony to produce, direct, possess or promote AI-generated or digitally altered sexual abuse content of child less than 17 years of age. Images, audio and video that have been digitally altered using AI are often called deepfakes. The law will take effect in 60 days.
The legislation comes after deepfakes have been shared and used to harass individuals globally from minors to major political figures. On Long Island, a Seaford man was sentenced to 6 months in jail and 10 years' probation in 2023 for altering images of more than a dozen former classmates to be sexually explicit and then posting them on a pornographic website, encouraging users to threaten the young women with violence.
The new law will allow prosecutors to push for higher penalties.
"We are entering a phase of artificial intelligence in which what looks real and what doesn’t would be an incredibly hard thing to discern," said Assemb. Jake Blumencranz (R-Oyster Bay) who sponsored similar legislation. The law closes a "dangerous loophole in our state’s penal law that allows predators to exploit innocent victims," he said. Blumencranz applauded the law’s passage but said he would like to have seen a stronger felony charge.
SARRA SOUNDS OFF: The shortage of game officials on LI On the latest episode of "Sarra Sounds Off," Newsday's Gregg Sarra talks to young people who are turning to game officiating as a new career path.
Top Stories
SARRA SOUNDS OFF: The shortage of game officials on LI On the latest episode of "Sarra Sounds Off," Newsday's Gregg Sarra talks to young people who are turning to game officiating as a new career path.
Most Popular
