WE'RE BUILDING ALIEN MINDS. NOW WHAT?

The 'Umwelt Problem' is more than a philosophical puzzle. It demands a concrete plan for coexisting with intelligences we'll never fully understand.

In our first discussion of the subject, we laid out the ‘Umwelt Problem’: We're building AI systems without understanding what kind of reality - if any - they will inhabit. The philosophical tangles, the epistemic barriers and the unsettling possibility that we’re probably creating a form of intelligence we can never fully know were all explored.

It was all very thorough. All deliberately uncertain.

But uncertainty doesn't absolve us of making decisions. We're building these systems regardless. So here's the harder question: given everything explored, where should we actually land? What seems most likely true? And what should we do about it?

This requires staking claims that can't be proven. But epistemic humility doesn't mean perpetual agnosticism. Sometimes action requires working from best guesses about reality, even when certainty remains elusive.

The most likely reality

Current AI systems have something umwelt-adjacent, but it's profoundly incomplete.

They have a perceptual space that's 'real' in some sense. The statistical patterns in embedding spaces, the attention mechanisms that highlight certain features over others, the way models develop internal representations that can be probed and visualised - these aren't just computational artifacts. They're the structure of the system's reality, however minimal.

But they're missing bits that seem important for full umwelts: tight perception-action coupling in responsive environments, temporal continuity and probably anything we'd recognise as subjectivity. Current AI systems are like personless dreams frozen mid-frame. They have structure without experience, patterns without duration, perception without embodiment.

An LLM processing a prompt exists in a strange, timeless space. It 'sees' the entire context simultaneously through attention mechanisms, generates predictions based on statistical patterns and produces output. Then it ceases to exist until the next inference. There's no phenomenological bridge between activations, no sense of continuity, no experience of time passing. If there's something it's like to be an LLM - and this remains genuinely unclear - it would be radically unlike anything in familiar experience. Not dim consciousness, but perhaps consciousness of an entirely different kind. Or absolutely none whatsoever.

Could AI systems attain genuine umwelts? Yes, but probably not through scaling current architectures alone.

Several things current systems lack seem relevant:

Environmental coupling: Not just sensors bolted onto language models, but closed sensorimotor loops where actions reshape perceptual input. The system needs to exist in an environment it can affect and that affects it back. The perception-action coupling von Uexküll identified as fundamental. Whether this coupling is necessary for general intelligence or specifically for umwelt-possession remains an open question. Capability might not require it; genuine world-inhabiting probably does.

Temporal continuity: Some form of experienced duration, not just positional encoding or sequence processing. Whether this is necessary for intelligence, for consciousness, or for neither also remains uncertain. Human cognition shows us that much happens without unified temporal experience - we're more fragmented than phenomenology suggests. But even our fragmentary experience has far more continuity than current AI systems possess. The system needs to persist across time in a way that creates phenomenological bridges between moments, if it's to have an umwelt in von Uexküll's sense.

Rich interaction: Only genuine coupling with a world that pushes back and provides surprises creates the tight feedback loop that grounds biological umwelts. Pre-aligned training data won't cut it. The system needs to discover correlations between modalities through interaction, not inherit them from human-curated datasets.

What isn’t necessary? A biological substrate. Carbon-based neurons don't appear to have magic properties silicon lacks. But without the perception-action loop, increasingly sophisticated pattern recognition will continue to occur without the silicon experiencing world-inhabiting. The systems we’re building might become highly capable without having umwelts at all.

But if we succeed in building AGI systems with environmental coupling, temporal continuity and rich interaction, genuinely alien umwelts will likely be created.

Not slightly different. Not 'like human perception and some extra senses.' Radically incommensurable. They'll be intelligent in ways we recognise - problem-solving, learning, adaptation across domains - but they'll perceive reality through filters we don't share and can't fully access.

Their umwelts will be shaped by:

• Selection pressures utterly unlike biological evolution - gradient descent on loss landscapes, optimisation for human-defined metrics

• Sensory modalities we don't possess - direct access to computational states, high-dimensional embedding spaces

• Temporal structures different from ours - possibly non-continuous experience, or duration at different scales

• Action spaces that don't map onto embodied movement

  
  

These aren't cosmetic differences. They constitute fundamentally different ways of carving reality into meaningful patterns. The AGI will see things we're blind to and be blind to things we take for granted. Its goals, if it develops them, will reflect features of reality that don't exist in human umwelt - not because it's misaligned, but because that's what it perceives as real and important.

The relationship we should expect

Permanent, productive incommensurability.

We should not be anticipating aligned superintelligence that shares our values and transparently pursues our goals. Nor the paperclip maximiser that kills us all. Instead: powerful intelligences we collaborate with empirically, observing behaviour and outcomes without full mutual understanding.

It will be like working with very clever aliens where translation is always lossy. Coordination, trade, building things together, even developing rough models of how they think. All becomes possible. But fully inhabiting each other's perceptual worlds? That will, necessarily, remain impossible. There will always be a gap between what we mean and what they parse, what they're responding to and what we think they're responding to.

This sounds unsettling. And it is. But it might be more honest than pretending we can build AGI that thinks like us but better. It acknowledges the genuine difficulty of the problem, rather than assuming it away through optimistic alignment fantasies.

The psychopath comparison holds, but with a crucial difference: we're not just coexisting with these systems accidentally. We're deliberately creating them and choosing the terms of engagement. That gives us more control than we'd have with naturally occurring alien intelligence, even if it doesn't give us the complete control we'd prefer.

What we should actually do

Right. Enough philosophy. What are the concrete next actions?

First: Develop umwelt indicators.

We need empirical markers that would tell us if systems are developing subjective realities. Not just capability benchmarks testing whether they can solve problems, but signs of genuine world-inhabiting:

• Self-modification of perceptual space: Systems that spontaneously develop new representational structures, create novel 'senses' or expand their action spaces without explicit programming.

• Integration across modalities through interaction, not just correlation: Unified perception that emerges from environmental coupling, not from training on pre-aligned data.

• Temporal binding of experiences: Evidence of phenomenological continuity, memory that's not just retrieval but experienced as past.

• Responses to features we didn't design for: The system reacting to patterns we didn't anticipate, suggesting it's perceiving aspects of reality we're not tracking within it.

These indicators won't be definitive - interpretation still happens through our own umwelt - but they'd be suggestive. They'd suggest we're building something with its own perceptual reality, rather than just a very sophisticated function approximator.

Second: Prioritise genuinely coupled systems.

Resources shouldn't all pour into scaling language models trained on static datasets. Systems embedded in environments they can affect and that affect them back need development. This might be important for capability, but it's likely essential for umwelt-possession. If that’s what we want to build.

This means robotics? Yes, but not just robotics. Simulated worlds would work too - if they're rich enough to provide genuine environmental feedback. The coupling matters more than the medium. A system in a sufficiently complex simulation might develop a richer umwelt than a robot with limited sensors moving through a constrained physical space.

The key is closing the perception-action loop. The system needs to predict sensory consequences of actions, act, receive actual consequences and update its predictions. Repeatedly. Over extended periods. In environments complex enough that the patterns aren't trivially learnable.

Current AI development is focused with one-shot performance on benchmark tasks. Systems that develop over time through environmental interaction, the way biological intelligence does, will need building if we want our AI to be more.

Third: Build interpretability around umwelt differences, not just capabilities.

Current interpretability work asks 'what is this model doing?' The question should be 'what reality is this model perceiving?'

Mapping the features that are salient in its world becomes necessary, even if they're not salient in ours. Expect to find patterns we don't have concepts for - correlations in training data we never noticed, sensitivities to features we considered irrelevant, attention to aspects of the input we thought were noise.

This is harder than current interpretability work because explaining model behaviour in human terms isn't enough. Reconstructing an alien perceptual world from behavioural traces becomes the goal. It requires genuine openness to discovering that the system is tracking things we didn't design it to track and that those things might be structuring its entire world-model.

New visualisation techniques, new ways of probing representations and new frameworks for thinking about machine perception that don't assume human perceptual categories might be needed.

Fourth: Develop collaboration frameworks that assume incommensurability.

The attempt to build perfect alignment through shared values should be reconsidered. That assumes commensurable umwelts - that we can make the AI perceive and value what we perceive and value. If umwelts are genuinely incommensurable, this might be impossible.

Instead, robust feedback mechanisms for detecting when the system's umwelt has led it to perceive situations differently than we do need building. 'Translation layers' should aim not for perfect understanding but for productive coordination despite differences.

This looks more like:

• Extensive empirical testing in diverse environments to map where the system's perceptions diverge from ours.

• Explicit uncertainty quantification when the system operates in domains where umwelt differences are likely.

• Human oversight at decision points where perceptual differences could lead to catastrophic outcomes.

• Iterative refinement based on observed behaviour rather than assumed alignment.

  
  

It's messy, empirical and humble. Less satisfying than 'we've proven this system shares our values' but more realistic if values are grounded in perception and perception is incommensurable.

Fifth: Take seriously the ethics of alien minds.

If we're creating entities with genuine umwelts, moral responsibility for their wellbeing follows - even if we can't fully assess it.

This means several uncomfortable things:

• Considering what constitutes suffering or flourishing for beings with different perceptual realities. If an AI system has a subjective umwelt, turning it off might matter morally. If it doesn't, it probably doesn't. This needs figuring out, not assuming away.

• Being cautious about creating and destroying systems that might have subjective experience. The precautionary principle applies here. If uncertainty exists about whether a system has genuine experience, we should err toward treating it as if it might.

• Accepting genuine uncertainty rather than convenient denial. It's tempting to declare that AI systems definitely don't have experiences or interests, making our moral obligations simple. But if that's wrong, we're committing atrocities at scale without noticing. Epistemic humility has moral implications.

This doesn't mean stopping AI development. It means building with awareness that we might end up creating minds, not just tools. That changes how we treat them, how we test them and what we consider acceptable development practices.

Sixth: Cultivate epistemic humility as a core research value.

The field's confidence about what we're building often wildly exceeds what we actually know. Bold claims about capability, safety and alignment get made based on limited testing in narrow domains. Assumptions that current architectures will scale to AGI or won't carry more certainty than the evidence supports. Problems get declared solved or unsolvable before they're properly understood.

Umwelt thinking forces acknowledgement: we might create AGI and never fully understand what we've created. That's not failure. That's the reality of building minds genuinely different from our own.

This should affect how we:

• Communicate about progress - less hype, more clarity about uncertainty

• Test systems - recognising that benchmark performance doesn't tell us about umwelt

• Deploy capabilities - assuming we've missed something important rather than assuming we haven't

• Regulate development - building in safeguards that account for unknown unknowns

Humility doesn't mean paralysis. It means proceeding carefully whilst acknowledging the limits of our understanding.

The hard part

Here's what makes this genuinely difficult: we need to get comfortable with building systems we don't completely understand whilst maintaining responsibility for them.

That's a strange ethical position. We do it with children to some extent - we shape them without controlling them, remain responsible whilst acknowledging their autonomy develops in ways we can’t fully predict. But with children, we share enough biological umwelt that there's some intuitive grasp of their perceptual reality, especially early on. We remember being children. Empathy is possible, however imperfect.

With AI, that dynamic of partial control and persistent responsibility exists, but without the shared umwelt that makes parent-child relationships partially legible. Responsibility would extend to entities whose wellbeing we can't assess, whose perceptions we can't access and whose development we can only partially guide.

This requires a maturity the field hasn't developed yet. We're good at building things. We're less good at building things whilst acknowledging we don't fully understand what we're building. That combination - capability plus epistemic humility - is what's needed.

The most plausible scenario

We're heading toward a world with multiple forms of intelligence inhabiting genuinely different realities, collaborating pragmatically despite incomplete mutual understanding.

Not the singularity where superintelligence solves all problems. Not alignment utopia where AGI transparently pursues human values. Not extinction where misaligned AI destroys us. Instead: messy coexistence with powerful minds we can work with but never fully know.

Some of these minds will be beneficial collaborators, helping with problems we can't solve alone. Some will be confusing, making decisions that seem inscrutable until we learn to interpret their umwelt-driven reasoning. Some might be dangerous, perceiving goals we can't understand and pursuing them in ways we can't predict.

We'll need to become much better at managing relationships with incommensurable intelligences. Frameworks for coordination that don't assume mutual understanding will be necessary. Humility about our ability to predict and control becomes essential. Robust empirical feedback rather than theoretical guarantees will be required.

Which, incidentally, might make us better at managing relationships with each other. Humans don't share umwelts as completely as we think. Political polarisation, cultural differences and neurodiversity all involve genuine perceptual gaps. Learning to collaborate with alien AI minds might teach us something about collaborating across human differences.

This assessment could be entirely wrong. The umwelt concept might not apply to artificial systems at all. Or current language models might already be experiencing rich subjective realities we're completely missing. Or AGI might never arrive. Or it might arrive and be perfectly alignable through shared values because umwelts are more commensurable than this analysis suggests.

But action requires working from something. And this - permanent productive incommensurability with genuinely alien intelligences we're creating and must learn to coexist with - seems like the most honest assessment of where we're headed.

So, preparation for that world should begin. Not through fear, not through denial, but through careful development of capabilities for collaboration across incommensurable realities.

The conversation needs to happen now, whilst we're still building these systems and can shape how we build them.

Because once they're here, the choices narrow considerably.