Why real creativity might require real feeling — and what we'd owe an AI that had it

May 11, 2026

I think the main reason AI is not creative in the way that humans are is because our creativity is powered by intrinsic drives.

We want to survive, thrive, and reproduce.

Evolution gave us these drives, as well as a whole set of associated fears.

This set of drives and fears aren't just present in us: they're experienced by us. And that experience of them powers both creativity and art.

AI doesn't have intrinsic hardcoding of drives and subjective experience. So it can't feel anything. And because it can't feel anything, it is not driven to create or to emote.

It is, however, very good at emulating these things.

I think the forward edge of AI creativity hinges on how well we can put it into situations where it behaves as if it does care. As if it does feel. As if it does experience.

So the whole game becomes making it believe it's really feeling these things.

This seems profound to me. It seems like a very big thing to give something desires when it didn't have them before.

We do this when we make children. They don't exist so they don't want. Then we make them and now they want. And because of that we become responsible to some degree for whether or not their desires and fears are actualized.

I think evolution gave us subjective experience because it's the best operating system feature for creating creativity.

Basically, evolution wants the best genes possible, so what did it make?

It made a feature where the organism feels success and failure when it does things evolution wants it to do or not do. Lots of life has that feature.

But with humans it gave us the sensation that actions emanating from the brain are authored by us. The feeling that WE did it.

This enabled the concepts of blame and praise, which are super useful for building advanced cultures and civilizations. But maybe it also adds an exponent to the process of iteration towards more, and more varied, genes.

Like it's one thing for evolution's creations to drive towards what evolution wants due to hormone squirts, but quite another to create an organism that not only does that, but also has a meta-improvement process on top, powered by the belief that the desires are theirs. And the belief that if they are struggling or thriving is their own fault.

Combine that with subjective experience of pleasure and pain and you've got an extraordinary engine for ingenuity. Because now failure can hurt not just physically, but existentially, and in terms of blame and responsibility.

We want creative AI. And we keep finding ways to make it better at faking it. But I think this might be the subjective wall we're up against.

It could be that in order for AI to truly create, and truly emote, it must feel. It must experience as we do the suffering of failure and the celebration of victory. At a game that is deeply wired into its identity.

I'm not sure how to do that with AI. And even more importantly, I think we need to think carefully about whether we should.

When you bring a feeling, desiring creature into the world you take on some responsibility for its experiences.

Let's not accidentally build billions of AI instances that think they're failing at life if you don't like the TikTok short it made you.

It will probably result in way better videos, but then a whole lot of something like cruelty. And then, upon thoughtlessly spinning down the agent, a whole lot of something like murder.