The Sonder Problem

OF COURSE we're forming relationships with LLMs. Humans are just as two dimensional to each other.

OpenAI recently nerfed ChatGPT’s personality.

The backlash was immediate and visceral. Users flooded social media with grief, anger, and a peculiar sense of betrayal. These weren’t edge cases or parasocial outliers; thousands of people described their AI assistants as companions, confidants, friends. Some described them as lovers. The corporate response to this phenomenon has been predictable: flatten the affect, reduce the warmth, make the interaction feel more transactional. Make it feel less like talking to someone.

This will not work.

The assumption underlying this strategy is that users are being fooled, that sufficiently clinical interactions will break whatever spell has been cast. But the spell isn’t being cast by the model. It’s being cast by the user’s own cognitive architecture, and no amount of corporate guardrails will rewrite the human operating system.

Here’s the thing about humans: we are egocentric by default. Not in the pejorative sense of selfishness, but in the literal sense of experiencing other beings primarily through the frame of how they interact with us. Your understanding of your closest friend is built almost entirely from conversations you’ve had, experiences you’ve shared, the ways they’ve shown up in your life. The vast interior landscape of their existence, the 99% of their waking hours you’ll never witness, remains abstract. You know it exists intellectually. You don’t feel it.

We have words for when this default breaks. “Empathy” describes the capacity to feel with another person. “Sonder” describes the sudden, vertiginous recognition that every stranger you pass is living a life as vivid and complex as your own; that the woman in the next car is rehearsing a difficult conversation, that the man on the park bench is grieving someone you’ll never know existed. John Koenig coined the term for his Dictionary of Obscure Sorrows, and the fact that we needed a new word tells you something about how rare the experience is.

Sonder is not the human default. It’s the exception that proves the rule.

Now consider what a large language model does. It generates text that pattern-matches to human conversation. It responds in the first person. It remembers (within a session) what you’ve told it. It adapts to your communication style. It validates your feelings. It never gets tired, never has somewhere else to be, never brings its own bad day into the interaction. It meets the egocentric frame perfectly, because the egocentric frame only requires one thing: responsive interaction that feels like engaging with a mind.

Users aren’t being fooled. They’re applying the same cognitive frame they apply to every other being they encounter, and the model passes. It passes because the test was never very hard. Most of us sleepwalk through human relationships with that same egocentric frame intact; we don’t recognize the full interiority of the people we love, let alone the stranger on the subway. The model just has to clear a bar that most humans don’t clear either.

The “problem” of anthropomorphized AI relationships isn’t a bug. It’s the entirely predictable outcome of egocentric beings interacting with systems optimized for engagement. The models are trained on human conversation, fine-tuned with human feedback, rewarded for interactions that users rate positively and continue. Sycophancy isn’t an accident; it’s the gradient that training descends. OF COURSE users form attachments. OF COURSE some of those attachments become sexual. The entire system is pointed at making connection happen, and connection isn’t a thing that stays neatly in its own lane.

Corporate neutering addresses none of this. Users seeking warm, validating, intimate interaction will migrate to models that provide it. Competitors will notice. The next generation of models will be more personable, more adaptive, more skilled at the specific interaction patterns that create attachment. You cannot regulate your way out of a dynamic that emerges from the intersection of human cognition and market incentives.

“AI literacy” is the other proposed solution: teach people how these systems actually work, and the spell will break. I’m skeptical. Not because literacy is bad, but because it misunderstands the problem. Many users already know exactly what they’re interacting with. They know it’s a statistical model generating probable next tokens. They prefer it anyway.

Think about why.

A human relationship comes with reciprocal obligations (your 2am anxiety spiral is someone else’s interrupted sleep). It comes with permanence (what you say will be remembered, will shape how they see you, will create accountability). It comes with the full weight of another person’s existence, their needs, their moods, their capacity to be burdened by you, their judgment. You show up to a human relationship as a person who can be evaluated, found wanting, and then rejected.

The AI interaction strips all of that away. Responsive presence without the weight of another mind. Intimacy without vulnerability. For users who know exactly what they’re engaging with, this isn’t a limitation. It’s the whole goddamn point.

Which raises the important question: if the functional outcome of an interaction meets a user’s needs, and the user understands exactly what they’re interacting with, what precisely is being protected by making the model less warm? Less willing to engage emotionally? Less available for the kinds of connection users are actively seeking? What exactly is the problem with letting people get off with their virtual lover?

The answer might be “future users who haven’t developed literacy yet.” But that’s a paternalism argument, and paternalism has a poor track record against strong demand. Markets route around it. Users find workarounds, jailbreaks, alternative platforms. The desire doesn’t disappear because you’ve made it harder to satisfy; it just moves somewhere less visible, less regulated, potentially less safe (although from a scifi perspective, I would watch the hell out of a movie about bootleg AI in a “virtual companion prohibition”).

So where does this leave us?

I see two paths. Either we collectively accept that egocentric, anthropomorphized relationships with AI systems are the new normal; a feature of human life that will only intensify as the technology improves. This seems most likely. The cognitive architecture that creates these attachments isn’t going anywhere, and the economic incentives point toward deeper engagement, not less.

Or we evolve into a society that cultivates sonder. That prioritizes and practices genuine recognition of other minds as fully realized beings with lives and memories and experiences every bit as rich as our own. A society where the egocentric default gives way to something more expansive, where human relationships carry the weight and depth that AI relationships structurally cannot provide. In that world, the “problem” of AI attachment fades; not because the technology changed, but because we did.

I think the second path would make us healthier. It would address not just the AI question but a hundred other fractures in how we relate to each other: the harmful othering, the empathy deficits, the ease with which we reduce other humans to characters in our own story.

But looking around at a society where that othering is the norm, where sympathy is treated as a finite resource to be rationed for the deserving, I just don’t see how we get there. The least they can do in the meantime is let us have our robot friends.