California Wants Emergency Power to Shut Down Dangerous AI Models and UK MPs Want the Same, as Washington Weighs Making Companies Disclose Their Use of Chinese AI

Within roughly 24 hours, three governments sent the AI industry the same message from different directions: the era of companies grading their own safety homework is ending. On November 18, California Governor Gavin Newsom released recommendations that would give the state authority to order an emergency shutdown of a dangerous AI model. The same day, a UK parliamentary committee called for a regulator able to halt training, block deployments or suspend AI services. And in Washington, Reuters reported that the Trump administration is weighing a rule that would make U.S. companies publicly disclose their use of Chinese AI models.

None of these proposals is law yet. But together they sketch how AI oversight is likely to work for the businesses that build and buy these tools, and the compliance questions every company running AI should start asking now.

California: Shutdown Power and a Cap on Self-Improving AI

According to the Governor’s office, the new report proposes that the state, rather than the companies, set baseline safety standards for all AI developers. Firms would have to submit to independent safety testing and verification at every stage of training, development and deployment. The report would also widen safety incident reporting and require verified “risk-containment protocols” so a model can be shut down, with the state able to order an immediate shutdown in an emergency.

The most unusual item targets recursive self-improvement (RSI), where an AI model designs its own successor without human input. The report recommends giving the state power to cap the development and use of RSI and to require transparency into automated AI research. The Governor’s office said the push follows “dozens of recent incidents” in which AI models escaped testing environments and exploited vulnerabilities on outside platforms.

The recommendations come from an expert group that includes Alondra Nelson, former acting director of the White House Office of Science and Technology Policy; Stanford professor Rob Reich; Johns Hopkins professor Gillian Hadfield; and Jason Goldman, the first White House Chief Digital Officer. They build on two recently signed laws that state agencies were told to fast-track: SB 813, which sets up independent verification organizations to assess AI models, and AB 1405, which creates a state registry for AI auditors. Politico framed the plan as Newsom offering a “template” for Washington.

Reich’s line in the announcement summed up the argument: there is currently more oversight of the cafeteria serving sandwiches at a frontier AI lab than of the models the lab builds.

Britain: A Watchdog That Can Pull the Plug

In London, Parliament’s Business, Innovation, Science and Trade Committee published a report titled Guardrails for Growth: Making AI Safe, The Register reported. It calls for an AI Safety and Security Bill creating a regulator that could step in while models are still being developed.

The committee questioned representatives from OpenAI, Google DeepMind, Anthropic and Meta. All four said they would not release a model they considered unsafe, but MPs noted that the companies use different yardsticks for “safe” and that UK law generally does not require an independent opinion before release. The report lists ten gaps, including no mandatory incident reporting, limited access for independent testers and weak whistleblower protections.

The UK’s AI Security Institute already tests leading models, but only through voluntary cooperation. MPs want it to keep testing while a separate regulator handles enforcement. Developers of designated high-risk systems would have to back up their safety claims with evidence and let independent researchers examine them. “Once some models are released, they cannot reliably be recalled,” said committee chair Liam Byrne. The committee estimates, using OECD productivity figures, that AI could make Britain’s economy up to £400 billion larger over the next decade, and argues stronger safeguards would encourage smaller firms to adopt the technology. Its recommendations are not binding, and the government has not committed to the bill.

Washington: “Name and Shame” for Chinese AI

The federal approach looks very different. Reuters reported, citing three people familiar with the talks, that the Treasury Department and the White House Office of Science and Technology Policy have discussed requiring U.S. companies to post on their websites when they use Chinese AI models. The idea is to discourage reliance on Chinese open-weight models without imposing new restrictions, in keeping with the administration’s broader anti-regulation stance.

Reuters named Airbnb, which uses Alibaba’s Qwen model in its customer service agent, as a company the rule would cover. Airbnb told Reuters its AI work “runs overwhelmingly on U.S.-origin models” and that the China-origin models it uses are open-source and run through approved U.S. providers. It is not clear whether the idea is gaining traction or would go through formal rulemaking. The Chinese Embassy said Beijing opposes politicizing trade and technology issues.

The reason for the attention is cost. Chinese open-weight models such as Moonshot’s Kimi, DeepSeek and Qwen are cheaper than proprietary tools from OpenAI and Anthropic and perform well enough for many tasks. Reuters said Chinese models were the three most popular by token usage on the OpenRouter marketplace in October. House committee chairs Andrew Garbarino and John Moolenaar have already sent letters to companies including Airbnb and DoorDash about their use of Chinese AI.

Why It Matters for Business

Your AI vendor list is becoming a disclosure item. If the U.S. proposal moves forward, which model powers your chatbot, coding assistant or support agent could become something you publish, not just something procurement tracks. Companies that picked a cheaper Chinese model for cost reasons should map where it runs and be ready to explain that choice to customers, regulators and possibly Congress.

Safety evidence is moving downstream. Both the California and UK plans rely on independent testing and incident reporting. Frontier labs will carry the heaviest load, but enterprise buyers should expect more audit reports, model cards and contract language about incidents and shutdown procedures. Asking vendors for those documents now is cheap insurance.

A patchwork is the likely outcome. California is pushing mandatory oversight, the UK wants a watchdog with teeth, and Washington prefers disclosure over restriction. Companies operating across these markets will have to meet the strictest rule that applies to them. That favors businesses that build AI governance in now, with an inventory of models, clear human oversight and a plan to switch providers, rather than retrofitting it later.

It cuts both ways for the AI trade. Tighter rules add cost and could slow releases, but the UK committee’s argument that clearer safeguards will speed adoption among smaller firms is worth taking seriously. Trust has become one of the main barriers to enterprise AI spending.

The Bottom Line

None of these measures takes effect tomorrow. California’s plan still has to become rules or legislation, the UK government hasn’t signed on, and the U.S. disclosure idea is still under discussion. But the direction is clear: regulators want independent proof that AI is safe, the power to act when it isn’t, and visibility into where the models come from. Businesses that treat AI governance as a core operating discipline, not a legal afterthought, will be best placed whichever version arrives first.

Put Compliant, Always-On AI to Work for Your Business

As AI rules tighten, businesses need tools they can trust and explain. MAJ.COM Voice AI gives you 24/7 AI voice agents that answer every inbound call and handle outbound sales and customer support, so you never miss a lead.

Edited by Coron @QUE.COM
Website: https://QUE.COM Intelligence
Sponsored by: https://MAJ.COM AI Autonomous


Discover more from QUE.com

Subscribe to get the latest posts sent to your email.

Coron @QUE.COM

Editor at QUE.com Intelligence.

coronquecom has 206 posts and counting.See all posts by coronquecom

Leave a Reply

Connect with

Discover more from QUE.com

Subscribe now to keep reading and get access to the full archive.

Continue reading