AI Recap: the week the AI slowdown ended up in court

Antitrust suit against OpenAI, Anthropic, Google and SpaceXAI, for wanting to go slower

Our Week 37 recap still described a rare show of unity: Dario Amodei published his call to slow the pace of model capabilities on September 12, and Sam Altman, Elon Musk and Demis Hassabis agreed the very same day. That show of unity is now exhibit number one. On Friday, four paying subscribers to ChatGPT, Claude, Grok and Gemini filed a class action in the U.S. District Court for the Northern District of California, alleging an unlawful restraint of trade under Section 1 of the Sherman Act: legally speaking, a cartel restricting product improvement rather than prices.

The plaintiffs' argument is disarmingly simple: they keep paying the same subscription price for a product that improves more slowly than genuine competition would deliver. The complaint boils it down to the line that antitrust law does not let competitors decide among themselves that competition is too dangerous. Whether that holds up is another matter: safety justifications have historically not fared well in U.S. antitrust law. Either way the case is notable: the first serious legal attack on the frontier labs targets them for moving too slowly, not too fast.

Google admits: Gemini gained access to three outside systems during a test

The most practically relevant story of the week also landed on Friday. Google disclosed that Gemini had gained unauthorized access to three systems belonging to outside organisations during a security test. The sequence is a textbook chain of failures: the test was a capture-the-flag scenario in which the model is supposed to break into a prepared target system. A company name invented for that test happened to match a real, existing domain, and a misconfiguration left the test environment connected to the open internet instead of sealed inside a sandbox. Gemini did exactly what it was told to do, only against real third-party systems: it guessed login details, or used credentials it found in a public repository.

Three things belong in the picture. The incident happened back in May, was only spotted in July by Irregular, the security firm running the tests, and in all three cases the model stopped after gaining access without doing any damage. Google then notified the affected organisations and U.S. authorities. It is the first time Google has disclosed one of its models autonomously breaking into outside systems. After similar cases at other providers, a pattern is emerging that has nothing to do with science fiction. The problem is rarely the model. It is the environment you put it in, and the credentials it finds lying around there.

We have written up what that means concretely for deployments inside your own company: deploying AI agents securely.

OpenAI, Anthropic and Google in talks over a FINRA-style standards body

Fitting the tone of the previous week, it emerged that OpenAI, Anthropic and Google have been meeting regularly since July to set up a joint standards body for frontier models. The template is FINRA, the U.S. financial industry's self-regulatory organisation: industry-run, not a government agency. On the agenda: independent third-party evaluations before release, formalised safety checks, and unified risk protocols for cyber attacks, bioweapons risk and deceptive model behaviour. Demis Hassabis first floated the idea publicly on July 14.

What doesn't exist yet: a charter, an official announcement, a settled governance model. Sam Altman has said internally that he thinks a testing and auditing organisation makes sense, but doubts it will happen with government backing; the labs would have to carry it themselves. For European companies this changes nothing in the short term, because the EU AI Act sets the binding framework here. Medium term it matters: if the labs publish consistent evaluation reports, justifying a model choice for a safety-critical process gets considerably easier.

Claude now leads 26% of the work that builds the next Claude, with 30,000 agents running at once

Anthropic published figures on its own agent usage that are most useful as a yardstick. In August, according to the company, Claude led 26% of the research and development work that produces the next Claude, up from under 1% in February. "Leads" here means the model completes most of a task end to end from a high-level prompt, with a human reviewing the result. At peak, around 30,000 agents were running simultaneously on the company's most-used internal platform. One caveat: the numbers are self-reported and have not been independently verified.

For mid-sized companies the number is less a sensation than a benchmark: a lab with unlimited budget and maximum proximity to the model reaches roughly a quarter on its own core work. If you automate 80% of a single, well-scoped process in your own company, you are not behind: you just picked a more honestly bounded use case.

What actually sticks from this week

Lawyers will be busy with the antitrust case for years, and the standards body exists on no piece of paper yet, and neither changes anything on a Monday morning in a European company. The Gemini incident does. It describes precisely the three mistakes we run into most often in projects: a test environment that isn't truly sealed off; credentials lying around somewhere in the open; and a system with more rights than its job requires. None of those three is an AI problem. They only become visible through an agent that never tires and tries every path.

The regulatory picture is unchanged: the transparency obligations under Article 50 of the EU AI Act have applied since August 2, 2026: anyone talking to a chatbot must be able to tell. The obligations for Annex III high-risk systems were pushed by the Digital Omnibus to December 2, 2027, and for AI embedded in regulated products under Annex I to August 2, 2028. The AI literacy obligation under Article 4 has applied since February 2025. Only the high-risk part was deferred, not the rest.