CAN THE MODEL FORGET?
There’s a tree on an island near Seattle that has swallowed a bicycle.
A boy left the bike in the woods in 1954 because he didn’t like it, and never gave it another thought. Nearly 60 years later, a reporter asked him about it. ‘I think the tree owns it now,’ he said.
He’s probably right. You won’t get it out now without taking a saw to the tree.
Customer data can end up in an AI model’s training a bit like that. Nobody quite means it to. But it’s much easier to teach a model to stop saying something than to make it forget. And that gets awkward the day a customer asks you to delete theirs.
In ‘Can the model forget?’, we compare two kinds of memory: the notebook you can open and edit, and the reflex that arrives before you’ve thought to look anything up.
A fact trained into a model is much closer to the reflex. Or the bike. You can’t point at it and lift it out. And researchers have got ‘forgotten’ information back by changing the prompt, or simply by asking the same question a few more times.
So where you keep the information becomes an architecture decision. Once it’s trained into the model, you may have made a promise you can’t keep: that you can take it out again if a customer asks. But keep it in the notebook – a separate store you control – and you can open it, correct it or tear the page out.
Which leaves one of the essay’s practical questions for any team using agents. If that request came tomorrow, could you name every place their data lives, logs and backups included?
And has any of it been trained into a model? Because by then, the tree owns it.
Read the full essay on Integrated Intelligence.
‘Can the model forget?’
