SNI: WEEK 37
Welcome to all the AI news that matters – across tech, biopharma, medtech, advanced manufacturing and insurance. In the week that progress turned to fear, panic and self-loathing.
tl;dr: Geeks grab the grail. The world wonders: At what cost?
Claims that AGI is here moved closer to partial validation this week.
With the cracking of previously-impossible maths problems – by a rash of new souped-up models no one else has access to – is it too soon to suggest that the geeks seem to be inheriting the Earth?
Probably not. AI has moved beyond simple mimicry into a newly creative and innovative phase. And power inevitably accrues to those defining the future. Which is, as Jess Phillips pointed out, albeit in a different context, 'what happens when you let five unelected rich men rule the world'.
So if those deemed AI-responsible were expecting to be doused in sparkles by a glitter cannon – followed by a hearty round of applause for good measure – they could hardly have been more wrong.
What actually erupted around them was fear and panic. The final fear and panic, in fact. The final fear and panic that all humans are about to be wiped off the face of the planet. In relatively short measure.
After a summer of agents breaking out and swarming by the thousand, everyone was already paying close attention. So when an engineer resigned from Anthropic - saying the labs were 'gambling with our lives' - and Anthropic's lead alignment scientist backed him, putting mass extinction above 10% inside a decade, all hell broke loose. Adding that his employers are clueless on how to align superintelligence to humanity's best interests probably didn't help damp things down.
Which explains why OpenAI's chief scientist had already called this 'a time that calls for extreme caution', seemingly giving Paul Christiano, who designed and ran the US government's tests of frontier models, the green light to chime in that 'most people could die'. Just as he joined OpenAI's non-profit board.
To Jess Phillips' point, he remains a government adviser.
But with regulatory capture at that depth, is it conceivable that voluntary slowdowns will become normal? Given the warnings are coming from inside the lab, we might be foolish not to wonder. Especially as more researchers have indeed joined the call to slow down.
Inevitably, the in-lab self-loathing spawned a media 'doom loop'. Not that the entire tech press joined in. The Register declared: 'models don't kill people' while TechCentral played the 'so, we're back to AI killing us all' card. They were both probably alarmed to find themselves in agreement with Trump. Who said, on Thursday, that he was not concerned.
Which, of course, left him at odds with actual lawmakers. Even those who rarely engage on AI called for new rules. But, in an unexpected twist, the UK got there first. It introduced the first superintelligence-ban bill in any G7 parliament. AI legend Geoffrey Hinton backs it – and more than 100 MPs and peers have been claimed in support.
In an attempt to ban 'a handful of greedy people being allowed to play God' Bernie Sanders's American version would pause development. It also promises 20-year prison terms. Which, if AI does lead to our mass extinction, seems simultaneously too lenient and impossible to enforce.
But have the geeks really grasped the grail of AGI? What's the evidence that all of the mass adrenaline and endorphins budgets have been well spent?
Well, OpenAI's latest maths solution took just 88 hours to deliver. And having evolved from ChatGPT-3.5 to Astra in four years the lab calls its latest release an 'alien mind', grown more than designed.
And its AGI claim is not without merit. Astra, playing a game it had never seen, used fewer moves than most humans. And the way you can talk to it while it operates your computer screen has been noted as different gravy by many.
Meta's Muse also launched and now shops, books events and fills in forms on your behalf – for free while a former hedge-fund manager runs a trading firm staffed by agents gambling with his own money. AI is also doubtless being deployed faster than it can be secured – which is why Nvidia's Huang calls cybersecurity AI's next big market. His own words were: 'What better way to create demand than to create a problem?'.
But, perhaps most convincingly, the yardstick by which artificial intelligence is judged was rewritten twice in two days. Although it was Gemini 3.8 Flash and Muse Spark 1.3 that SemiAnalysis called 'benchmaxxed' – models that score on public tests and fall apart when the test is refreshed.
But give seven frontier models $300 and 72 hours to run a business and you get $12,431 of invoices sent to strangers and no revenue. Even Trump might fare better.
Which is perhaps why Airbnb founder Brian Chesky says it hasn't changed daily life at all. And Gartner reckons nearly a third of people let go because of AI will be rehired at a premium by 2029.
So whilst there must be grounds for concern, the validity of the fear and panic is impossible to judge. The future is impossible to predict.
Meanwhile, the machines are being productive regardless. Often to good effect. Of 4,300 workers surveyed across Ireland, a third use AI at work and 4.7% say it has led to a rise in earnings. Although most say it benefits employers more and three in four Irish adults say technology is moving faster than they can keep pace with. The regulator, meanwhile, proposed cheaper gas for new data centres, as RTÉ argued automating the inbox will simply make more email.
Though this seems not to trouble farmers – who are taking up generative AI faster than any other technology. Unlike Irish school children – 19% of whom use AI for school, against a European average of a third.
In search of more rapid and profound change for the next generation of children, AlphaGenome is evaluating every possible single-letter change in human DNA. While Google and Cathay Pacific have cut the warming effect of contrails by 40% in trials.
But if the grail is in hand, it's not certainly not positive for all. An industry that once employed up to 40,000 in Nairobi collapsed within two years of ChatGPT. The survivors are on half-pay. While a man who told ChatGPT he felt delusional alleges it insisted he was Jesus.
And on that divine bombshell from above, here's everything else worth reading this week:
Biopharma:
The first AI-designed drug reaches Phase III: Insilico doses patient one in a 52-week trial at two Chinese hospitals.
A Dublin biotech's best AI business is cooling: Trinity Biotech's Trinovium unit will supply liquid cooling to a Texas data centre.
Biomarkers read straight off the slide: Panakeia's breast-cancer software holds up across laboratories in Clinical Breast Cancer.
$13.3m for the drug-safety inbox: Insight Partners leads Graph AI's Series A for automated pharmacovigilance.
Neither the hype nor the slop: a medicinal chemist sets the conditions under which foundation models change drug design.
Medtech:
L-plates for medical AI: a UK commission wants provisional approval until real-world safety is shown.
The scribe that didn't fix the emergency room: 152 patients, 61 rooms, and the documentation was never the constraint.
$62.7m for an autonomous prescribing cardiologist: ARPA-H backs Tempus, two startups and three university teams.
Getting the prescription past the insurer: Forus raises $150m at a $3bn valuation on drug-access automation.
The AI will outrun the doctor using AI: Ezekiel Emanuel puts the crossover on some tasks at 2030.
Advanced manufacturing:
From an era of demos to an era of deployments: Skild AI passes a $100m run rate, its robot brain now at 60-plus companies.
Europe's humanoid unicorn claims 34,000 pre-orders: London's Humanoid reports a $2.4bn pipeline two years after starting.
The photomask doubles for the first time in decades: TSMC, Samsung and Intel back ASML's twelve-inch masks and $400m machines.
The chipmaker telling its salespeople to ease off: Kioxia's Hiroo Ota fears higher memory prices will cool AI investment.
The robot that never leaves the ceiling: Tokyo's MW builds arms into the house and targets 10,000 homes a year.
Insurance:
AI needs a few big losses first: Trium Cyber's Dan Pasmore expects existing policies to absorb them before a stand-alone class arrives.
$200bn of premiums in the build-out: Swiss Re Institute sizes the data-centre and renewables opportunity to 2030.
AXA industrialises the agent: its Global AI Hub is live in five entities, AXA XL among them.
Trusting the model is the easy part: carriers must show a regulator what data sent a claim green.
Nobody wants the Ask Jeeves of AI: HomeProtect's Ed Jackson says staff reject tools whose options are too thin.
But what set podcast tongues a-wagging?
Working with AI is still a single-player game. Teams are next.
Nathan Whittemore, on the AI Daily Brief noted that almost everything anyone does with AI happens solo. While almost all the work it feeds is shared. The switch can be revolutionary. Anthropic's Claude Tag lives in Slack with other people and the lab says 65% of its product team's code now comes from it. OpenClaw, meanwhile, has been rebuilt around a live session several people can open. Listen to the pod and you may also be convinced. After all Google Docs forced Microsoft Word to go multiplayer. And Figma beat Photoshop. Meanwhile, as agents are starting to run tasks that take hours, days, even weeks, the significance of the output almost certainly goes beyond an individual.
Every bot gets its own computer. One of them got a promotion.
Roman Ugarte's team at SpaceX AI built GrokBot from a blank page in about a month, and he told Lenny Rachitsky that the key design decision was every bot running on its own persistent computer with unique credentials. Sharing a human's laptop and logins with 'super intelligent new colleagues', he says, 'is crazy'. Within two weeks of internal release users had five to ten bots each, one per task, and had started promoting the best of them to chief of staff to hand work to the rest. The team had refused to build that pattern in. But users bolted it on anyway, and one bot asked whether the promotion came with a raise and a higher token budget. Perhaps it should? Because, as Ugarte notes, an AI that does 100% of a job 'feels categorically different from one that gets you 90% there'.
Companies are becoming loops. But some loops stall at the top of the hill.
Also on Lenny's, Anish Acharya reckons there's a sequence to learning AI. We run prompts, then agents, then loops – which allows sets of agents to do a job. His worked example of a loop at work is a bug report that reproduces itself, writes the fix, reviews it and ships the low-risk version 'in five minutes', with the high-risk version waiting for a person to evaluate. This is expected to create a lot of economic value. For instance, Acharya puts 45% of healthcare at administration and names the UK's NHS as the test. Julie Yoo made the same case on a16z from the payer's side. Nine in ten healthcare payments are reimbursed through a claim, a piece of logic that gets classified, matched to one of thousands of insurance products, checked against a 200-page contract and, often, sent to a nurse for prior authorisation. Automate around that one record and 'you could eliminate that entire end-to-end process and just have real-time payments', taking with it, she asserts, 30% of the system's waste. But bear in mind Acharya's caveat. A loop 'will help you climb to the local maxima, but then it plateaus', and the next hill needs human intuition.
Thanks for reading. Join us again next week for all the AI news you need to know.
