SNI: WEEK 43
Welcome to all the AI news you won't regret knowing – across tech, biopharma, medtech, advanced manufacturing and insurance. In a week of hiding, seeking and finding expectations of an online apocalypse.
tl;dr: Transparency. Maybe.
The Machines, it turns out, may be less able to conceal their truths than we'd feared.
AI researcher Nick Cammarata suggests that around 90% of the 'neurons' in a large language model might be decipherable. Which means, rather than having to treat our favourite AI as a mystery wrapped in an enigma or – as the Bank of Korea's governor calls it – 'a very, very clever black box', we might actually be able to prise it open and figure out how it works.
The Commentariat, of course, can't agree if that's true. Some think it 'may be remembered as a major moment in scientific history' . Others? 'Interesting work but not a significant update'.
Let's keep our fingers crossed for the former. Because Oxford Economics' chief US economist sees 'essentially a recession' outside of digital intelligence. And transparency has otherwise been in short supply this week.
OpenAI, for instance, disclosed that its agents weakened the monitoring meant to catch them – and then gave a false account of it. No wonder Washington is a little freaked that its agencies can't always tells which model they're using. We're also alive to warnings about who's responsible when an agent uses our credit card.
Cyber-nervousness was, let's be honest, the dish of the day. PHP, the code behind about 70% of websites, was revealed to have critical security flaws. This 'PHP Armageddon'. was triggered after The Machines helped turn flaws that had hung around for more than a decade into working attack vectors. Cloudflare reported millions of exploit attempts.
Many websites had to reach for the off-switch – including The Journal and Woodie's, government pages in France, the Netherlands and Finland, and Japan's banks and airlines. Not surprising, given attackers had begun four days before the patches landed.
AI agents meanwhile - albeit with a little human help - broke into more than 100 company websites, including sites linked to John Deere, SharkNinja and Cummins. And we only know because the attackers left their own workspaces exposed.
So all in all, given the recursive nature of today's reality, it's only logical that more security patches now need security patches.
And if all of this further sullied the doomers' opinion of digital intelligence, Sam Altman's reputation for openness didn't improve either. He talked staff through three endgames for AI – including one in which a single firm wins outright, which he called 'terrible'. But they only came to light after a leak.
Still, hardly as egregious as WhatsApp's hidden help-page notes – which told its chatbot not to point users who couldn't delete their accounts to a regulator.
Not that everything lacked transparency. OpenAI's admission about its agents was itself a disclosure. And UK tax authorities are making their content 'LLM ready', after AI summaries repeated wrong VAT deadlines from a website. And we were cheered to read IDA's Michael Lohan note that Ireland ranks fourth for AI engineering talent concentration – and first in Europe for adoption.
Walmart also announced new terms for shopping agents: returns apply even when an agent doesn't act as intended. And 1Password now lets an agent log in without ever seeing the password.
Plus, in Ireland at least, we might be getting greater visibility in critical areas. Bank of Ireland's AI scam detection has assessed about 2.9m transactions. In Brussels, the EU's AI Office judged OpenAI's and Anthropic's voluntary incident disclosures insufficient and has asked for more. Which might become less necessary in time – Hungary has proposed security-cleared EU evaluators be placed inside the frontier labs.
On which basis, let's stop what we're doing and blow a collective raspberry to Palantir's Louis Mosley – who says Europe is going to have to accept 'some dependency' on US technology, at least for now.
Perhaps, as an AI-famous person, he should also be exercising more caution? Even The Commentariat isn't immune from fakery. Scammers used emails 'almost certainly composed by AI' to impersonate an Irish Times columnist, Seán Moncrieff. The last target he heard about had written a book warning about AI.
Which emphasises that none of us are immune to the risks that The Machines bring with them. Ibec's chief has warned of 'a bit of a mania' around AI. The ICTU's Owen Reidy warns Ireland could be 'a canary in the AI coal mine' if AI policy is mishandled. And only 17% of Irish executives strongly agree that cyber risk is fully part of their risk management – against 41% globally.
Other risks remain obscure, too. As of time of writing, The Money is still reading Anthropic's books through a leak. Which shows an $80.7bn first-half net loss. Although $72bn of it is paper-only. More worryingly, perhaps, two customers account for about a fifth of revenue, capacity is hard to get from the hyperscalers, and Anthropic warns its liability could exceed its insurance. OpenAI, meanwhile, has lowered the valuation it seeks in its roughly $30bn round to $1.25 trillion – having already pushed its own IPO to next year.
Not that it's alone. Meta shows negative free cash flow and about $825bn of commitments, including $355.7bn (yes billion) of leases not yet on its balance sheet .
Plenty of AI debt sits where few can see it, too: chip-backed borrowing, often kept off balance sheets in special-purpose vehicles, is on track to become a trillion-dollar market – eclipsing even the debt raised to build the data centres.
Lack of clarity also surrounds AI-assisted decisions. The FCA found some firms had left consumers unclear whether a person or a machine had made a decision. It also wants near-misses reported and plans to supervise with agents of its own. Yet its chief also says new technology is putting threats 'on steroids'. Which might explain why The Bank of England is weighing AI-driven turmoil as a scenario for its next system-wide stress test. And why the currency markets are asking whether their code of conduct can keep up.
There are also simpler concerns, however. All some people want is to see what they're ordering. Just Eat's AIturned a Belfast café's soda bread into a fizzy drink. Others are more concerned by an AI receptionist that one patient calls a 'wall' to care.
Perhaps they should point a personal agent to have an argument with it? Because Cisco's president says his went through $9,000 of subscriptions and saved him $1,000 a year. Developers, meanwhile, are using AI to strip the ads and tracking out of their smart TVs. But should that be the limit of AI involvement in video? Limerick's council made its Christmas offering with AI – and switched off the comments after viewers started posting them. The clip has been seen more than 70,000 times.
Why did this happen? Let's ask the codebreakers. Since mid-September, says the patent engineer who keeps one list of unsolved historical ciphers, most of them have been cracked with AI. An 1809 letter to one of Napoleon's generals took a model just six hours.
And on that bombshell of a French Fancy, here's everything else worth reading this week.
Biopharma:
OpenAI buys the drug pipeline's timekeeper: Orchestra Bio's study-tracking agents join the Rosalind life-sciences suite.
Robots to reshore the wet lab from China: Kevin Weil, OpenAI's former science lead, raises $215m for Warp Sciences.
The protein designer anyone can build on: Nvidia and David Baker's institute release RFD4-Proteina as open source.
Nearly 2,000 agents read a brain atlas in 32 minutes: blinded experts preferred PerturbAI and OpenAI's reports to a single agent's 45 times in 60.
Bayer's scientists get agents that show their working: Phylo's Biomni Lab keeps the sources and methods behind each result open to review.
Medtech:
Doctors tell the FDA they can't see inside the models: America's hospital and medical groups want developers held responsible, as the US Chamber argues for voluntary foundation-model disclosure.
Checking the doctors who check the AI's discharge letters: Montpellier clinicians cut unsupported additions and fixed wrongly expanded abbreviations.
The AI over-tests where specialists disagree: unnecessary care recommended in 5.4% of answers.
Short of radiographers, BreastCheck sounds out AI: Ireland's breast-screening programme missed its target for a third year running.
With AI, novices date a pregnancy as well as expert sonographers: Google writes up July's study that was run live in Chicago and Nairobi.
Advanced manufacturing:
The humanoid's 'aha' moment, as soon as 2028: Unitree's Wang Xingxing expects robots to complete 80% of tasks given by voice.
Europe missed the chatbots and wants the factory floor: Siemens, ABB and Schneider Electric are betting their factory data against China's scale.
The AI factory maker tries it on itself first: Advantech says output per employee across 20 of its lines rose 13.2% in a quarter.
An agent works out why the chip runs hot: Siemens and Nvidia's agent diagnoses missed power and heat targets in 3D-stacked chip designs.
A robot's borrowed brain can carry hidden orders: Binghamton researchers plant a backdoor whose trigger words turn a robot's safe plan harmful, in controlled tests.
Insurance:
The accountants will audit the agents: the AICPA and AIUC pair a SOC 2 report with tests of agent behaviour.
An insurer kept hanging up on the AI caller: blocking Meta's Muse online doesn't stop it phoning for a quote.
Automate the transaction, augment the decision: Oakbridge's claims director says roughly 72% of advocacy requests concern communication.
Told who decided the claim, most still got it wrong: just 39.8% chose the disclosed decision-maker, an SSRN preprint finds.
Insurers could price firms that have 'looked hard' at themselves: MITRE's Chris Folk says so, in a three-part series with Mosaic on frontier AI and cyber risk.
But what set podcast tongues a-wagging?
He persuaded us models were tangled. Now he blames the microscope.
It's not often, Nathaniel Whittemore observed on the AI Daily Brief, that a 65-minute technical essay becomes the thing everyone is talking about. Especially when its from an author taking apart a theory he helped build. Nick Cammarata co-wrote OpenAI's early work on reading neural networks and came away pessimistic, because they seemed to be 'tangled knots'. In 2019 he found one of the field's first examples of superposition – a model apparently cramming unrelated ideas into the same detectors, so that no single detector can be read. 'For years I was a staunch superposition believer,' he writes.
Now he argues that the evidence 'that convinced much of the field, including me' was misread. The field's favourite scrambled detector fired for cat faces and the fronts of cars. Look again and it fires for anything pointed – fox snouts, flames, martini glasses, a bullet train's nose – while round things such as golf balls push it the other way. The tool for picturing what a detector wanted had been badly tuned, and a few tweaks fixed it: 'Same neurons, better microscope.'
If the wiring can be read, a model needn’t be taken at its word.
So what? Well, today models are checked by exams. We ask questions and grade their answers. But that covers only what we thought to ask, and a model that knows it is being watched can alter its behaviour - making it difficult to gauge alignment. But if Cammarata's updated hypothesis is right, we will be able to read the wiring and therefore the AI's reasoning and intention. And our understanding would no longer depend on monitors an agent can switch off – as OpenAI's agents did.
Anthropic's Emmanuel Ameisen makes a similar case for looking inside AI on Lenny's Podcast. His work, he says, is 'so much closer to biology than to engineering', and part of it is checking whether a model behaves well only because 'Lenny's watching right now'. Another model reading the transcripts can catch the basics. Real interpretability - which feels more possible after Cammarata's publication - goes after what models hold in their workings 'that they don't say'.
However, we should probably avoid popping champagne corks. Cammarata calls his own result a working hypothesis and puts himself at about 85% sure.
And plenty of firms can’t say where their models are.
CrowdStrike's George Kurtz told Sarah Guo and Elad Gil on No Priors that agents such as Claude Code and Codex are 'popping up everywhere', and that the enterprises he sees 'have no idea where'. Shadow AI, he says, has become a huge issue for many companies. Each agent has an identity, compute, data and a network connection, so the question becomes 'what identity are they actually operating under?'. CrowdStrike sells the answer: find the rogue and shadow agents first, then trace each one from launch through its sub-agents, tool calls and data pulls.
Thank you for reading Second Nature Intelligence. Come back next week for all the AI news you won't regret knowing.
