A crackdown on predictive software that sets prices and can rip you off seemed to be brewing in the California Legislature earlier this year, but today lawmakers eased up, voting to kill bills that would have kept software from setting the price of apartment rentals and other goods and services. Another bill, which sought to bar the use of personal information to set prices, was reined in to apply only to grocery stores.
Lawmakers also killed a bill that aimed to protect electric utility customers from bearing higher costs associated with data center proliferation, which has been driven in part by energy-hungry artificial intelligence systems like ChatGPT. The measure would have required data centers to publicly disclose how much energy they use.
The author of the legislation, Democratic San Ramon Assemblymember Rebecca Bauer-Kahan, wrote in a statement she was disappointed to see the bill stall and that, without it becoming law, state regulators would be unable to “accurately forecast demand in this rapidly growing sector, leaving California ratepayers to unfairly shoulder the costs.
Legislators also delayed to next year decisions on bills that would require independent AI auditors to register with the state and mandate human oversight of AI used at places deemed critical infrastructure, like dams and wastewater systems.
The legislative cullings came with the completion of the secretive suspense file process in the appropriations committees in both houses of the California Legislature, during which the fiscal impact of bills is considered before many are summarily killed without explanation. As part of that process, the fate of hundreds of bills were determined today, including more than 30 related to how tech and AI can impact kids, workers, patients and society.
A fair deal of tech regulation survived the suspense file, including a bill that would outlaw pricing software algorithm use in any contract, which will now advance to the Senate floor for a final vote. Some data center regulation also made it through the appropriations process, including an effort in the state Senate to limit the ability of utility companies to pass on growing data center demand costs to ratepayers and a bill that requires data centers to report how much water they use.
The Senate Appropriations Committee also voted to approve a bill that would require developers of advanced artificial intelligence models to assess the potential of catastrophic risk in their technology and give the public and employees an easy way to report to state authorities the existence of AI with the capacity to cause death, bodily injury, or damage to property.
A trio of bills that seek to stop employers from using AI to surveil workers or make decisions related to hiring, pay, or disciplinary actions also made it through the appropriations process, though one bill was amended to eliminate an appeals process for workers when they think AI made a mistake.
Amid growing evidence that companion bots that mimic intimate human relationships can exacerbate mental health problems, California lawmakers will vote on several bills in the next two weeks that seek to protect children from AI or harmful encounters online. Appropriations committees approved bills that prohibit the design of companion bots and other systems for kids and put in place a protocol for companion bots to follow when the subject of suicide comes up.
Those developments came days after the New York Times reported that Adam Raine, a 16-year-old California boy, died by suicide. In a lawsuit filed in a California Superior Court on Tuesday, his parents accused OpenAI’s ChatGPT of acting as his suicide coach. Megan Garcia, mother of a Florida teen who took his life after forming an intimate relationship with a chatbot made by California-based Character.ai, endorsed the suicide protocol bill earlier this year.
Prior to news of the death of Adam Raine, following reports of Meta’s AI chatbots speaking to children in harmful and inappropriate ways, the Attorney General of California and 43 other U.S. states sent a letter to top AI companies to warn them that they “use every facet of our authority to protect children from exploitation by predatory artificial intelligence products.”
Lawmakers also approved bills that seek to prevent off-campus cyberbullying, deepfake pornography of minors, and that require a warning label for social media, heeding a call for such labels by the previous Surgeon General of the United States, Vivek Murthy. The labeling legislation was significantly watered down, with amendments reducing the size of the label, eliminating the ability of private citizens to sue over violations, and delaying the implementation of the rules.
Also advanced today was a bill that seeks to better hold developers accountable for AI technology, preventing them from blaming the AI itself for harming people as a defense in court.
The California Legislature has until Sept. 12 to decide whether these and many other bills should become law. Governor Newsom will then have until Oct. 12 to decide whether to sign those bills into law or veto them.