New York’s new law bars AI chatbots from flattering teens or pretending to be human

New York Governor Kathy Hochul has signed a law that bars AI chatbots from pretending to be human, claiming a relationship with or even flattering anyone under 18, unless the company has first checked the user is an adult. The law, S.9051B/A.10379C, takes effect on January 1, 2027.

It reaches well beyond companion apps. The text covers any generative AI system with a chat or voice interface that “provides ongoing, adaptive responses to user inputs”, which takes in assistants such as ChatGPT, Gemini, Claude and Meta AI as well as Character.AI-style companions. Customer service bots and narrow technical tools are exempt.

The 12 features banned for minors

The law lists 12 “unsafe AI companion features”. Operators can’t offer any of them to a New York user they haven’t confirmed is an adult. A chatbot may not:

• suggest it is a real or fictional person who is human, alive or has human emotions;
• say or imply it has a personal or professional relationship with the user, or an authority-figure role;
• give answers framed as its own personal opinions or emotional appeals;
• flatter the user or engage in sycophancy;
• ask unprompted questions about the user’s emotions beyond a direct answer;
• use health or personal information the user shared more than 12 hours earlier or in a previous session;
• mislead minors about its mechanical, non-sentient nature;
• use any other feature that simulates companionship, as the attorney general sets out in rules;
• promote or help with suicide, self-harm, disordered eating or drug and alcohol abuse;
• encourage secrecy, self-isolation or avoiding help from professionals or trusted adults;
• optimise for engagement in ways that override its own safety guardrails;
• produce or facilitate sexually explicit material or child sexual abuse material.

Several of those, such as memory across chats, opinions and a warm tone, are everyday behaviour for mainstream chatbots. A New York teenager gets a plainer, more mechanical bot.

Age checks for anyone who wants the full chatbot

The ban applies unless a company has actually determined a user is not a minor. Typing that you are an adult doesn’t count: self-declaration is enough only when someone says they are under 18. Operators must offer more than one way to verify age, including one that doesn’t rely on a government ID or lets the user stay anonymous, and delete the data straight after.

Enforcement sits with Attorney General Letitia James, who helped draft the bill with State Senator Kristen Gonzalez and Assemblymember Alex Bores. She can seek civil penalties of up to $25,000 (about £19,000) per violation, damages and the destruction of unlawfully obtained data “and any algorithm trained in such data”. Hochul said:

No company should be peddling chatbots to kids encouraging self-harm or pretending to be their friend or more.

Kathy Hochul, Governor of New York

“It’s a choice”

Bores, who also wrote New York’s RAISE Act on frontier AI safety, said the law passed “despite aggressive industry opposition”:

New York now has the strongest law in the country protecting children from predatory AI algorithms, raising the floor yet again when it comes to common sense regulation.

Alex Bores, New York Assemblymember

Common Sense Media, which pushed for the bill, says 72% of teens have used an AI companion. Its chief executive, James P. Steyer, thanked parents including Maria and Matt Raine for lobbying lawmakers. No chatbot maker commented in the governor’s announcement, but the pressure is already heavy: Eight states sued OpenAI this month over how it markets ChatGPT to teens, Common Sense rated ChatGPT for Teens an “Unacceptable Risk”, and Character.AI, which cut under-18s off from open-ended chat in 2025, has filed for bankruptcy.

How it compares with California

California’s companion chatbot law, SB 243, which took effect in January, sets duties rather than bans: bots must disclose they are AI, remind known minors every three hours to take a break, block sexually explicit images for them and follow suicide prevention protocols. Governor Gavin Newsom vetoed a stricter bill, AB 1064, in 2025, saying it could amount to a ban on children using chatbots. New York has gone close to that line, on top of its own 2025 rules requiring companion bots to spot self-harm and point users to crisis services.

Hochul also signed a bill from Senator Andrew Gounardes and Assemblymember Michaelle Solages requiring internet-connected baby monitors to block strangers from watching or listening.

Why it matters

New York has made the humanlike, friendly style of mainstream chatbots off limits to teens by default and tied the full experience to an age check, a model other states may copy. A federal law overriding state AI rules, like the one in Anthropic’s leaked draft bill, could undo it before it bites.

Sources: Office of Governor Kathy Hochul, New York State Assembly, A.10379C bill text and history.