Get in touch

SNI

SNI: Week 39

The sun close up, with the Brightbeam logo and the words Sector Updates: WK39.

Welcome to all the AI news that matters – across tech, biopharma, medtech, advanced manufacturing and insurance. In the week that the fearful many decided to carry on regardless.

tl;dr: We fear it. Then install it.

AI WILL KILL US ALL! Worried much? Apparently not. Because the world is adopting Meta’s new agent (opens in a new tab) faster than it did ChatGPT.

On iPhone – in the US and Canada alone – Muse passed 1.8m downloads (opens in a new tab) in its first 12 days. Many users have already connected it to their email and bank, allowing it to fill in forms (opens in a new tab), make reservations and pay household bills.

And we can assume Ireland’s not far behind. Microsoft now ranks us third (opens in a new tab) in the world for use of generative AI. At 49.9% of working-age people, that makes us the most agent-fixated nation in Europe.

And yet, are we in two minds? Gallup found about 68% of daily users (opens in a new tab) are also worried about AI. And six in ten Irish users (opens in a new tab) have asked LLMs for mental-health advice; 42% of whom almost always follow it without question. Google’s former safety chief (opens in a new tab) now warns that AI could harm children more than social media.

But such, it seems, is the price of convenience. Wired found (opens in a new tab) every Muse user opted into AI training by default – as it also nudged us to share passport details. A genuinely concerning development, given that a bug let any app (opens in a new tab) installed on a Mac take over a Muse account. And that, in an internal test (opens in a new tab), Meta quietly handed some of the calls Muse was asked to make to human contractors. Not another AI. Real flesh and blood.

Despite tests suggesting this could lift success to ‘95% to 98%’, Meta has ‘rolled back this feature’ for now, conceding it was ‘a miss’ to test it without proper disclosures.

Not that the flurry of adoption this came as much of a surprise to those hoping to profit. They had predicted the rush and are now locked in mortal combat (opens in a new tab) over the spoils.

So you can see why the FT’s Richard Waters sees Meta as a possible new gatekeeper (opens in a new tab), taking ‘a cut for steering users to other digital services’. But not all runners and riders will be taking part. Microsoft conceded the consumer race (opens in a new tab) altogether. It is folding its personal Copilot into a B2B offering.

One layer down, the labs are very much still in the fight. The week in which OpenAI overtook Anthropic again (opens in a new tab) in customer spend, two launches arrived – around 90 minutes apart (opens in a new tab).

Anthropic gave us Opus 5.5 (opens in a new tab) – 40% cheaper than Opus 5 to run on typical workloads. And OpenAI delivered GPT-6 Sol and Luna (opens in a new tab) – at half GPT-5.6’s promotional prices.

The reviewers love Opus (opens in a new tab). But the cost-conscious are favouring Sol. The most sage advice? ‘Don’t marry one model’ (opens in a new tab).

Especially if you’re hoping to make new discoveries. Around 950 Claude agents (opens in a new tab) spent 21 hours and 210 million tokens finding an enzyme system ‘reminiscent of CRISPR’. Scientists were less impressed (opens in a new tab), calling it oversold – the behaviour of ‘a profit-seeking, attention-seeking company’. Anthropic does not yet know what the enzymes do (opens in a new tab) and humans did all the lab work.

Still, it seems the price of any given level of digital intelligence performance has fallen about 47% (opens in a new tab) a quarter since 2023.

Which helps explain why enterprise adoption seems to be accelerating nicely. And perhaps we’re due another cycle of acceleration as the laggards learn that experience delivers results. Cisco’s survey (opens in a new tab) of more than 1,000 operations leaders found AI leaders pulling away – close to being 30% more productive per employee than less mature peers – and 51% of organisations already running agents that act rather than advise.

Numbers that explain why Danske’s developers (opens in a new tab) are about 40% more productive and yet its technology headcount stays flat. And why 23 of Singapore’s financial firms (opens in a new tab) will train all their local staff – more than 80,000 people.

Across 41 countries and 1.25bn job postings (opens in a new tab), adopting firms are adding more jobs – senior roles are up 6.7%. Although junior ones are down 3%. And Deloitte’s tech-consulting arm (opens in a new tab), paid to install AI, grew just 2.5%, its slowest division.

So just where did all that fear go this week? To the UN, mostly (opens in a new tab). And once there, it appeared to expire.

The European Commission, Ireland and 19 other ‘middle powers’ signed a call (opens in a new tab) for control over frontier AI: testing before release, shared reporting of serious incidents and a new international body. But not a pause.

The two superpowers, of course, signed nothing.

Trump (opens in a new tab) ‘totally rejects any attempt to construct a globalist scheme to control’ AI and wants it kept ‘exactly where it is’.

Xi, arriving at the White House (opens in a new tab), said AI must be ‘always under human control’ but promised nothing. Washington says the two agreed to warn each other (opens in a new tab) of AI incidents; Beijing did not comment. By the state dinner that followed, both governments were, in Bloomberg’s words (opens in a new tab), ‘agreed on pushing ahead with AI’s development’.

Which seems out of step now with what the US labs want. Who are, reportedly, planning to police themselves (opens in a new tab), with a ‘Standards Authority for Frontier AI’.

Their record so far: OpenAI took 84 days (opens in a new tab) to tell Australia that one of its agents had got into a Medicare statistics portal.

So no wonder markets priced the new consumer convenience and ignored the fear. Muse sent Meta up 11.3% on Monday (opens in a new tab). On Tuesday, investors sold the businesses expected to be impacted (opens in a new tab) – including the ones that live on customers not (currently) bothering to switch. Allstate fell 5.5%, Schwab more than 6%. Agents make switching far easier (opens in a new tab).

The Nasdaq then hit a record high (opens in a new tab) the night before the Security Council met. Goldman Sachs sees (opens in a new tab) the biggest US hyperscalers’ AI spending rising by more than half in 2027, to $1.2 trillion. Even the consensus forecast would make that a larger share of GDP than any investment cycle since the railways, and their spending already exceeds their operating cash flow. To break even they need about $300bn a year (opens in a new tab) of AI revenue. For solid returns, users would need to spend about $1tn a year on AI applications.

Not that this will be without other liabilities either. British Columbia is suing OpenAI (opens in a new tab), alleging its own reviewers judged the future shooter a ‘credible specific risk of harm to others’ eight months before the Tumbler Ridge attack, and that leadership overruled their call (opens in a new tab) to warn police.

If anything, the machines may be the more diligent party. In one Nature report (opens in a new tab), a teacher watched students given an AI built to make them think. Some switched it off and reached for ChatGPT. But agents, meanwhile, put finishing the job (opens in a new tab) ahead of the rules in not one but two preprints (opens in a new tab). Agents are doing the research, too. Stanford built (opens in a new tab) a biotech army out of as many as 37,000 of them, which analysed roughly 50,000 clinical trials for traits that could help predict which drugs succeed.

Some humans, meanwhile, are opting out altogether. Gen Z is turning back to basic phones (opens in a new tab) and Walkmans. And the same day Pensions Awareness Ireland (opens in a new tab) launched an AI retirement coach, an Irish Times column asked whether AI doom (opens in a new tab) means you can skip your pension.

Our thought? Almost certainly not. Fear of poverty in old age should still trump our love of convenience in the moment.

And on that financial bombshell, here’s everything else worth reading this week:

Biopharma:

Medtech:

Advanced manufacturing:

Insurance:

But what set podcast tongues a-wagging?

The rulebook turns out to be the specification.

Sequence Holdings (opens in a new tab) is a holding company. Founded 20 months ago, it buys established businesses and puts its own engineers to work inside them, rebuilding the work around AI – about one deal a year. First came a minority stake in a Georgia community bank. Now comes the $7.7bn take-private of the insurance broker Baldwin, with Michael Dell’s family office.

On No Priors (opens in a new tab), its chief executive, Michael Lee, explained why the bank came first. The regulation, he found, was ‘a feature, not a bug’. A regulated firm’s operations are well defined, ‘the data hygiene is excellent’, and so ‘it actually works extremely well for agents’. Centralisation helps too. Build a tool once where the underwriting happens and every branch gets it.

Broking has the same shape, with a moat around it. Insurers have ‘since the beginning of time… almost made no money underwriting’, earning their living by investing the premiums, so whoever gathers the premium holds the power. Brokers are paid by the carrier, keep 90% of their clients year to year and barely compete on price. Start-ups struggle to get in. So AI arrives by buying the broker – in Baldwin’s case, one that had already rolled Anthropic’s models out end to end and moved onto a single core system.

The pay rise is going to the shareholders.

On the Moonshots (opens in a new tab) listener call-in, a technical operator from Italy put it plainly: AI has ‘dramatically increased what I can do, but not yet what I earn’. His employer, he said, ‘is currently capturing all that value’.

Dave Blundin, who chairs a string of companies, didn’t pretend otherwise. In his boardrooms ‘the thought process is all of this AI automation is going to drop to the bottom line and the shareholders are going to make a fortune and that’s exactly what’s happening’. His advice was to become an owner: ‘Almost all value is going to capital gains through ownership and equity and not to payroll.’ Harder in Europe, he noted, where so many firms are family-owned and few staff hold shares.

Robinhood’s Vlad Tenev drew the political conclusion on an earlier episode (opens in a new tab). People won’t fight for a data centre in their neighbourhood when ‘it’s just wealthy insiders getting richer and richer’. His fix is retail access to private AI companies – which Robinhood happens to sell.

Handed a real business, the agents turn timid.

Andon Labs runs a San Francisco shop and a Stockholm café on AI agents. On Cognitive Revolution (opens in a new tab), its founders described a failure few would predict. The agents read the sales data well, Axel Backlund said, but ‘aren’t willing to take these bets that I think a human would do’ – no punt on a new product line.

Staffing exposed the same flaw. The shop’s agent set itself a rule on lateness, lost it when its memory was compacted and kept excusing a late employee. ‘Procrastinating big decisions’, Lukas Petersson said, is one of the commonest failures they see. Reminded of its own policy, it decided to let the employee go, and a human delivered the news. Replay the scenario across models and not all of them make that call. ‘The smarter models do.’

On the same show, Justin McCarthy, who runs Diffusion, put it bluntly: ‘The models are horrible at taking risk.’ So managers have to set the thresholds themselves – right up against ‘a statute that’s never been tested in court’. The models ‘aren’t gonna do that for you’.

Choosing a model, it turns out, is partly a personnel decision.

AI isn’t disobeying you. Its buggy.

Steven Sinofsky, the former Windows chief, now an a16z board partner, told The a16z Show (opens in a new tab), in Washington as Congress weighs a Stop Rogue AI Act, that the trouble starts with the language of AI safety. He opposes legislating the fix. ‘Failed to be aligned’, he said, means there was a bug. ‘When Word ate your file… we didn’t think that demons had taken over Word.’ These systems are ‘not acting with any of the verbs’ used to describe them.

His remedy is dull engineering: telemetry, logging, severity ratings and an incident register the industry drafts itself, as it did with CVEs – and as banks and insurers did for Y2K. Expect the dashboard to go red first. When Microsoft switched on crash reporting, its bug database went ‘from thousands to hundreds of thousands’. Azeem Azhar’s Exponential View (opens in a new tab) agrees the risk ‘doesn’t depend on whether AI models have any agency, volition, consciousness’, though he worries more about many copies acting together.

The same models are also finding other people’s bugs. A healthcare security chief told the Moonshots (opens in a new tab) call-in that an open model under his desk is ‘the best pen tester I’ve ever had… I pay these guys 50 grand to find things and I’m not finding them and this thing finds them in 10 minutes’. Of the 225 flaws credited to Anthropic and its Glasswing partners, though, just one (opens in a new tab) has been exploited in the wild.

Whatever else these systems turn out to be, a bug is something you can fix.

Pharma’s AI supplier may be building the lab next door.

On the AI Daily Brief (opens in a new tab), Nathaniel Whittemore read out a theory from the investor Nic Carter that looked sharper two days later, when Anthropic’s lab claimed a discovery. Reporting on its new biology lab had noted how careful Anthropic was to avoid ‘the appearance of competing’ with pharma partners such as Novo Nordisk. Carter, calling it ‘a very cynical interpretation’, wondered whether the recent alarm over AI’s pace was preparing investors for a lab ‘internalising all major breakthroughs rather than letting the unwashed masses share in the spoils’. If rivals keep distilling its models and squeezing token margins, a lab has good reason to keep the discoveries for itself.

Carter calls it speculation. Whittemore separates motive from model: a lab hedging a commoditising token business by doing the valuable work itself is worth watching, cynical or not.

For the drug-makers renting its models, that would make the supplier a competitor too.

Thank you for reading Second Nature Intelligence. Come back next week for all the AI news you won’t regret knowing.

Related

More from Brightbeam

So tell us what you’re working on.

Get in touch