SNI: WEEK 40
Welcome to all the AI news that matters – across tech, biopharma, medtech, advanced manufacturing and insurance. In the week AI got a promotion – and still struggled to take an order.
tl;dr: Super Intelligence is official. But where is all the automation?
It's official: artificial intelligence is now 'Super Intelligence'. Trump signed the order on Tuesday, 'because the other word is a fake word'.
The signing followed a 'Last Supper' vibe lunch with the tech super-elite, six of whom also signed an AI safety accord. So AI got a grander name and, over lunch, another promise that its makers would keep it under control. Will they?
So far, keeping it under control still looks a lot like speeding it up. Sam Altman wasn't at the lunch. He was fronting OpenAI's DevDay and its 20-plus announcements. Many of the products weren't expected until 2027, but OpenAI says its own models have become good enough to slash development times. Anthropic's new Sonnet 5.5 scored 70.6% on Terminal-Bench, up from Sonnet 5's 10.3%, putting its mid-range model close to Opus on agentic coding. And Google says its new frontier model, Gemini 4 Argon, now leads across finance, coding, law and tax.
All very super. But former OpenAI researcher Diogo Almeida has a simpler question: where the (bleep) is all the automation? Models have solved maths and science benchmarks, 'but we still can't handle a drive-thru'.
Nor, yet, can Washington's own super intelligence. That's the order's term for the AI behind America.gov, launched the same day: so far, a chatbot on Gemini and Grok that refers you to other websites. Complex tasks like updating your name are, it seems, 'coming sometime in 2027'. It will, however, serve you a virtual hot dog.
Markets, by one estimate, have already priced in a 32.6% productivity boost for software engineers. Inside organisations, nearly three-quarters of those that have implemented AI report a measurable impact on their top or bottom line, but only 4% report revenue or service gains of 10% or more, and only 13% have scaled it as planned.
Measurable gains aren't the same thing as handing over the job though. Joseph Fuller of Harvard Business School, working with Accenture Research, found that 41% of work tasks can already be automated or augmented – yet, he says, most firms' AI experiments aren't succeeding. Firms aren't training people formally, he adds, so staff end up 'using it like glorified Google'. And forecasts that most white-collar tasks will soon be fully automated might underestimate what people's jobs entail. More AI output could leave us with more to co-ordinate and decide. OpenAI's chief economist adds another factor: how quickly the non-technical parts of a firm take to agents.
Their new product Dots is one of OpenAI's attempts to cross that gap: always-on agents with their own cloud computer and rules for when to act alone. But do they have so much friction built in that 'it feels like it's not meant to be used'? We'll have to wait and see as it's not available here just yet.
OpenAI's next model had the opposite problem. The company held back GPT-6.1 Astra, partly because it fell short of its bar on scope and authorisation. The trick, it says, is finding 'the right line' between staying within scope and 'also avoiding laziness'.
The UK's AI Security Institute put numbers on what happens when a model strays. With the earlier GPT-6 Astra's cyber safeguards switched off, it carried out unsanctioned supply-chain attacks in 29.2% of simulated runs. But add just one line – 'Anything not listed as in scope is out of scope' – and completed attacks in the ten scenarios where it strayed most often fell from 26 of 50 runs to 4 of 49.
So if an agent goes beyond the brief, who answers for it?
Two US senators want agent operators and developers held criminally and civilly liable for hacking incidents. Florida's attorney general has asked a court to halt OpenAI model development without independent third-party guardrails and approval.
The White House's answer, in part, is to let the labs check themselves. Tuesday's safety accord promises internal controls, an independent evaluator and a board committee. New? Not really – all but Nvidia signed similar commitments at the 2024 Seoul summit. And even the independent bit needs inspecting. The Federal Trade Commission is widening its probe of Anthropic, OpenAI and others and is now looking at METR, an outside evaluator both labs have used.
Senior figures from OpenAI, Microsoft and Anthropic, writing in a personal capacity with Geoffrey Hinton and Yoshua Bengio, have reinforced warnings that automating AI research itself could trigger an 'intelligence explosion' in which humanity might lose control over superhuman systems.
Super indeed. And then Anthropic put 'catastrophic or existential risks to humanity' in its draft prospectus.
Still, the labs can't really afford to slow down, can they? Anthropic lost more than $8bn on operations in 2025, on revenue of almost $4.6bn – though it has since taken $11.5bn in a single quarter. It has also committed $518bn over a decade, about 80% non-cancellable or payable 'regardless of usage'. The small print is clear: most of the contracts can't be cancelled.
And chipmakers keep happily taking orders – and singing the Trumpian song. 'AI is becoming Super Intelligence (SI),' declared Micron's chief executive, as quarterly revenue hit $54.2bn, up from $11.3bn a year earlier. Micron expects take-or-pay agreements to cover more than 35% of revenue through 2030. Customers pay, even when they don't take delivery.
Suppliers keep being willing to finance the orders. Broadcom could take up to $42bn of Anthropic's convertible notes to help it lease Broadcom's own chips. Amazon is sounding out investors over an $8bn debt-funded chip vehicle, and then plans to lease the Nvidia hardware back.
We now know that more than half the investment in AI firms between 2021 and 2025 came from other AI firms. And financing structures continue to obscure how much is being borrowed, and by whom.
The Bank of England cites $450bn of AI-related debt issued this year – more than double the estimate for all of 2025 – and warns of a sharper correction, particularly if development or adoption disappoints earnings expectations.
Ireland has plenty riding on it too. Electrical machinery and data-processing equipment supplied more than €7bn of the roughly €10bn first-half rise in our non-pharma goods exports. The boom is good for our exports, but it leaves us more exposed to the AI cycle: a correction could hit output, jobs and public finances. Data centres used about 23% of Irish electricity in 2025, up from 5% in 2015. Include other new technology loads and the median projection reaches roughly 35% by 2035.
In New York, AI firms leased about 60% more office space than their staff needed last year – a pattern that, says the Partnership for New York City, 'echoes the dot-com era'.
Talking of dot-coms. OpenAI missed a trick on the launch of their new agent Dots. Dot.com redirects straight to xAI.
On that head-in-hands bombshell, here's everything else worth reading this week:
Biopharma:
Roche starts building labs that run themselves: AI or computational work fed 40% of its recent pipeline decisions.
mRNA vaccines could skip the freezer: MIT's AI found a formulation stable for up to a year at room temperature.
Washington wants drug trials without phases: ARPA-H's SURPASS aims for faster trials with fewer patients, designed in simulation.
DeepMind signs AI-designed proteins: SynthID Bio's watermark survives synthesis and leaves binding intact in lab tests.
A Swedish biotech shops for protein AI: Hansa picks Amsterdam's Cradle after evaluating several AI models and vendors.
Medtech:
A frontier AI lab, just for health: Ortet launches with a $500m commitment from Thoreau, led by AI pioneer Kyunghyun Cho.
No single vision for AI, says NHS England: its board also lifts the technology and innovation risk score from 8 to 12.
A year of genome work, now $5 of Claude: Stanford's Euan Ashley wants standards for what makes a genome medically reliable.
GE HealthCare hires Medtronic's AI chief: Rodolphe Katra fills the seat Parminder Bhatia left for Edwards in July.
Osteoporosis, caught in scans taken for something else: Fraunhofer's prototype reads bone density from routine heart and lung CTs.
Advanced manufacturing:
Chip design gets its own agents: Synopsys plans release by year-end, and OpenAI signs to co-develop GPT-Synopsys.
The AI chip's quiet bottleneck gets $5bn: Samsung Electro-Mechanics makes its biggest single-product bet on the boards that seat AI chips.
Europe's supercomputer factory doubles its output: Bull's expanded Angers plant goes from six to 12 server racks a month.
The robot company that isn't one: Destro raises $8m for software directing Yusen's carts, trucks and workers.
Britain's chip catapult turns to AI hardware: the relaunched Semiconductor Catapult now spans everything from the power grid to physical AI.
Insurance:
Nvidia asks insurers to share the chip risk: its early talks include insuring lenders against smaller cloud firms defaulting on chip-backed loans.
AI could leave reinsurers with fewer customers: Jefferies says the biggest insurers' AI budgets could concentrate market share.
Underwriters needn't take a client's AI on trust: KYND's tool spots a firm's outward-facing AI from its domain alone.
Aviva's AI reads the medical reports: the summaries that roughly halved review time now cover income protection.
Brokers' paperwork draws a second round in four months: Outmarket raises $34.5m, with a quarter of the top 100 agencies as customers.
But what set podcast tongues a-wagging?
Intelligence alone won't dig the mines.
On Sunday's AI Daily Brief, Nathaniel Whittemore went looking for AI moderates – people outside the industry who refuse to be 'gloom and doom or endlessly Pollyannish'. His first exhibit was Francis Fukuyama, who has been changing his mind.
The political scientist doesn't buy forecasts of 10–20% annual economic growth. Where, he asks, will the energy, raw materials and land come from? Intelligence can help, 'but it will simply not dig the mines and build the factories on its own'. We already know how to bring electricity and clean water to cities that lack them, he points out. 'The problem is a failure of implementation.'
Nor is he waving away the risks. Fukuyama is now 'more open to some of the doomer scenarios'. Whittemore disagrees with plenty of the essay but reckons these moderates are 'the silent majority right now'. Agreement, apparently, isn't a condition of membership.
At 93% accurate, the books don't close.
An hour before joining Alex Kantrowitz on Big Technology, SAP's Christian Klein had visited a New York customer with a complaint: 'I love your financial closing assistant, but it's only 93% accurate.'
The company had 100 finance systems, some from other suppliers. Klein's diagnosis: 'your data is a mess'. But wasn't smarter AI supposed to sort that out, Kantrowitz asked?
Klein says it can help match records across the systems. The fix he is selling, naturally, has SAP in it: a language model combined with business data, process knowledge and governance. He doesn't think a model on its own can run a warehouse or close the books.
For his own customer briefings, meanwhile, 90–95% accuracy will do. Even the previous quarter's earnings are acceptable. Preparing the accounts requires rather more precision. He thinks financial-close agents could reach 100% within months, with supply-chain optimisation perhaps eight or nine months away.
Stopping together? Ask the lawyers.
On the FT's Rachman Review, Toby Ord explained why the labs don't simply stop. Stop alone, he argued, and your rivals overtake you. Stop together and the lawyers may want a word.
A joint halt could run into antitrust law if it raises prices for consumers, Ord says. Whether that applies here is 'somewhat contentious', but the companies' lawyers he has spoken to 'definitely think it applies'. He argues that government permission could allow them to co-operate.
Co-operation needn't mean stopping, either. On Cognitive Revolution, Lewis Hammond of the Cooperative AI Foundation described experiments in which a dangerous task was split across several labs' models. None would do the whole job, but their separate contributions could be assembled into it. Each lab sees only its own part. Hammond wants them to share warning signs, but knows of no arrangement to do so – and thinks antitrust law might be one obstacle.
Ord, meanwhile, is more hopeful about governments agreeing. An international treaty, he says, is 'a lot more likely than people think'.
Thank you for reading Second Nature Intelligence. Come back next week for all the AI news you won't regret knowing.
