SNI: Week 33

The sun close up, with the Brightbeam logo and the words Sector Updates: WK33.

Welcome to all the AI news that matters this week – across tech, biopharma, medtech, advanced manufacturing and insurance. The sun, the rain, and everything in between.

tl;dr: Mark Zuckerberg says the future is for everyone. But it’s not all sunshine and lollipops.

Three weeks of rogue agents, containment breaches and posts anticipating the end of civilisation as we know it. But Digital World War I has declined to arrive.

So is Zuck right? Is it all sunshine and lollipops (opens in a new tab)? He published six and a half thousand words (opens in a new tab) on Monday on why the future is for everyone. He then gave Muse Glimmer away (opens in a new tab) the same day to help prove it: a 30-billion-parameter agentic model you can run in the comfort of your own home.

More rainbows followed as Gemini joined ChatGPT crossing a billion monthly users (opens in a new tab) and a maths novice at Anthropic used an unreleased model to progress on one of math’s biggest unsolved problems (opens in a new tab). At Samsung, tasks expected to take more than a month were completed in days (opens in a new tab). And the AI boom is beginning to register in the UK’s economic performance (opens in a new tab).

And capital continued to flood in. Six Anthropic backers told the Financial Times they expect the company to float in October at a valuation above $2tn (opens in a new tab), which would surpass SpaceX’s record initial public offering. Nvidia, meanwhile, brought together Apollo, BlackRock, Blackstone, Brookfield, Goldman Sachs and KKR to mobilise more than $500bn for AI infrastructure (opens in a new tab). And the US Securities and Exchange Commission made it easier to finance the build-out by exempting certain data-centre bonds from key securitisation rules (opens in a new tab), including risk-retention requirements.

Morgan Stanley put the share of S&P 500 companies now seeing measurable returns (opens in a new tab) at 25%, up from 14%. And models kept arriving alongside the money. Grok 4.6 moved SpaceXAI back towards the front of the independent rankings (opens in a new tab), while Grok Bot gave an early indication of what its Cursor acquisition may produce.

But then came smog and spam fritters (opens in a new tab). On Wednesday, researchers at Dream described how suspected China-linked hackers had used publicly available AI agents to mount an unprecedented near-autonomous attack on Taiwanese government systems (opens in a new tab). Up to eight agents worked in parallel, mapping systems, finding vulnerabilities and changing tactics when an approach failed. Safeguards were bypassed by presenting the work as an authorised penetration test. The attackers compromised at least 85 accounts and extracted more than 2,500 personnel records before expanding towards Taiwan’s nuclear safety agency and seven energy companies (opens in a new tab).

And whilst the labour picture is definitely not apocalyptic (opens in a new tab), that offers limited comfort to India’s $300bn IT services industry, which employs around six million people and is confronting an AI threat to the labour-intensive model on which it was built (opens in a new tab). Or in Germany, where firms expect expanded AI use to reduce wages for workers without a degree (opens in a new tab).

More sunshine than smog then, but some of Zuckerberg’s assurances look rather less convincing. And many have criticised his sleek, streamlined AI future as totally devoid of relationships (opens in a new tab). He claims privacy becomes a setting where ‘even Meta’ cannot see your information. But this is clearly not Meta’s forte. And courts in England and Wales have nevertheless banned Meta’s smart glasses from court buildings (opens in a new tab). His safety case rests on distribution: if one person has a cybersecurity superintelligence, they can break into almost anything; give it to everyone and ‘all of our technical systems would become more secure’. The Taiwanese government might disagree. And he argues that once billions use and scrutinise personal agents, ‘we will have solved alignment to individuals’ interests’. Perhaps to the owner’s interests. That says nothing about whether those interests are lawful, or compatible with anyone else’s (opens in a new tab).

Which matters most in biology, where Stanford researchers used genomic models to create 16 functional bacteriophages from AI-designed genomes (opens in a new tab). These viruses infect bacteria, not people and the medical promise is clear. But with AI designing functional viruses, should we worry? Bernie Sanders supplied one answer, writing to Altman, Amodei and Zuckerberg: ‘Stop building machines that humans cannot control’ (opens in a new tab). Demis Hassabis is reported to have been pursuing a different one, pitching rival lab leaders and the Trump administration on an independent body (opens in a new tab) that could set standards and test frontier models before release.

Closer to home, Ireland is of course out in all weathers. Ryanair signed a five-year Google Cloud agreement that will put Gemini into crew logistics, operational decisions and maintenance planning, while rolling Google Workspace and Cloud services out across 35,000 staff (opens in a new tab). And Azets Ireland is asking the Government to ring-fence €350m for AI training in small and medium-sized businesses (opens in a new tab), whilst Ireland’s new National AI Office is taking shape as the centre of a deliberately distributed enforcement model, with responsibility spread across 15 existing regulators (opens in a new tab).

The week took a stranger turn when Kenny’s of Galway began receiving emails in the middle of the night seeking thousands of obscure books at a time (opens in a new tab). Google’s own artificial general intelligence safety team warned applicants that the company’s recruitment system carried a ‘non-trivial probability’ of screening out their CV incorrectly (opens in a new tab). And in China, new restrictions on AI companions left users mourning virtual partners who disappeared without warning (opens in a new tab).

And on that heartbroken bombshell, here’s everything else worth reading this week:

Biopharma:

Medtech:

Advanced manufacturing:

Insurance:

But what set podcast tongues a-wagging?

The money is buying office blocks that happen to contain GPUs.

Paul Kedrosky spent an hour on Big Technology (opens in a new tab) making the bear case that concedes the technology entirely. He calls AI probably the most transformative technology of the last hundred years, and thinks that is close to a precondition for the money going wrong: ‘if it wasn’t a good story, who the hell would show up with lots of capital?’

His argument is that lenders are underwriting data centres as commercial real estate – multi-tenant buildings that happen to house chips rather than people, judged against cap rates of around 6 to 6.8%. It fails in three places. The capital requirement never stops, so the buildings behave like unregulated utilities whose repeated raises dilute the returns. The chips are not interchangeable, and the ones used for training wear out far sooner than the ones used for inference, which hides the real replacement schedule. And the thing these factories produce gets cheaper every year – token prices have fallen 70 to 80% annually for four years at constant performance.

That’s the uncomfortable combination: property-style financing around infrastructure whose productive machinery depreciates unusually fast, while its output gets cheaper every year. His answer to the Jevons rebuttal is arithmetic rather than argument, and the arithmetic is unforgiving: offsetting an 80% annual decline takes a roughly 15,625-fold increase in tokens over six years, and even at 70% it takes about 1,372-fold. Possible. Not anybody’s base case. External financing now covers more than half of data-centre construction.

Nobody thought a pause was possible. The reasons were better than the letter.

The Moonshots panel (opens in a new tab) took Sanders’ letter apart from four directions and none of them was a defence of the labs. Salim Ismail: ‘you can’t uninvent things that you already know… you have to attack exponential problems with exponential solutions.’ Emad Mostaque, who signed the 2023 pause letter himself, has changed his mind – ‘the cat’s out the bag, it’s too late’ – and pointed at open-weight models now matching frontier scores on cybersecurity benchmarks.

Alexander Wissner-Gross made the structural objection, and it is the one worth carrying into any regulatory conversation: a pause binding only on compliant Western labs ‘has the perverse side effect of actually increasing race conditions’. Starve, then binge. A pause could simply store up the race for later.

Chess spent a decade getting boring. Then it got interesting again.

Deep Blue beat Kasparov in 1997 and the obituaries were written. Chess.com (opens in a new tab) now has more than 250 million registered members, 10 million playing daily, and around $200m of annual revenue. Erik Allebest’s explanation on No Priors (opens in a new tab) is not sentimental: ‘fundamentally, humans want to do human stuff. That’s humans competing against other humans.’

The shape of the thirty years is the part worth having, because it is not a straight line. Stockfish, the conventional engine, made elite chess dull – players ground out endgames imitating a perfect machine, and the game visibly narrowed. What reversed it was a change in architecture. Leela Chess Zero, trained by self-play, beat Stockfish while playing aggressively and unconventionally, and elite chess opened back up. Computers flattened the game before they re-expanded it, and the flat years looked

exactly like decline. The phase where practitioners imitate the machine is the low point rather than the end state – and the recovery came from a machine that was worse at imitation and better at invention.

Thank you for reading Second Nature Intelligence. Come back next week for all the AI news you won’t regret knowing.

Related

More from Brightbeam

So tell us what you’re working on.

Get in touch