Summary
Regulates operators of AI companion systems that use AI, generative AI, or emotional-recognition algorithms to simulate sustained human-like relationships with users. Requires covered AI companions to include protocols for suicidal ideation, self-harm, and possible physical harm to others, provide crisis-resource notifications when those expressions are detected, provide recurring disclosures that the user is not communicating with a human, and file annual safety-protocol reports with the Attorney General beginning July 1, 2027. Authorizes AG enforcement and civil penalties.
Healthcare Implications
Not healthcare-specific, but relevant to mental-health-adjacent AI companions and consumer AI systems that may interact with users expressing suicidality, self-harm, or emotional distress. Operators must implement crisis-response protocols, provide human/non-human disclosure, report safety-protocol metrics to the Attorney General, and face civil penalties for noncompliance.
Operational Implications
- Operators must provide a clear and conspicuous notification at the beginning of any AI companion interaction and at least every three hours during continuing interactions that the user is not communicating with a human.
- Operators may not provide an AI companion unless it contains a protocol for possible suicidal ideation, self-harm, or physical harm to others and provides crisis-resource notifications as soon as those expressions are detected.
- Beginning July 1, 2027, operators must file annual reports with the Attorney General that include the number of safety-protocol activations and related metrics; the Attorney General must publish aggregated data.