This post contains no AI-generated text or images, but does discuss experiments I’ve done in the past with AI art generators. For more info, read my statement on generative AI.
On , Kipekee the reticulated giraffe was born at Brights Zoo in Tennessee. Kipekee is remarkable in that she has no spots. In fact, she seems to be only the fourth brown spotless giraffe in recorded history; the last known one was born in . White spotless giraffes — who have what’s called leucism — are slightly more common, but still exceedingly rare.
I first learned about Kipekee from Janelle Shane’s AI Weirdness blog, where she asked some image recognition models about pictures of Kipekee. Although the models she chose were fairly highly regarded for their image recognition chops, they struggled to identify and comment on Kipekee’s spotlessness coherently or reproducibly. Kipekee’s spotlessness had thrown them for a loop.
Janelle attributes the image recognition models’ strugglebus to three main factors:
– Janelle Shane, “AI vs a giraffe with no spots”
- AI does best on images it’s seen before. We know AI is good at memorizing stuff; it might even be that some of the images in the examples and benchmarks are in the training datasets these algorithms used. Giraffe With No Spots may be especially difficult not only because the giraffe is unusual, but because it’s new to the internet.
- AI tends to sand away the unusual. It’s trained to answer with the most likely answer to your question, which is not necessarily the most correct answer.
- The papers and demonstration sites are showcasing their best work. Whereas I am zeroing in on their worst work, because it’s entertaining and because it’s a cautionary tale about putting too much faith in AI image recognition.
That second factor — AI tends to sand away the unusual
— struck something of a chord. I wasn’t alone in this. Echoing that sanding factor on Mastodon, J. Rosenbaum, who researches AI's perceptions of gender, added:
– J. Rosenbaum (@minxdragon@wandering.shop)
Boom. 🎤 Now imagine that you are one of the edges that is sanded away.
As someone who’s visibly physically disabled, I don’t have to imagine.
My arms are shorter than most, and my left is noticeably shorter than my right. My hands bend inwards at a 90° angle at the wrists, and I have no thumbs. I’ve got craniofacial differences, and a hearing aid implant to boot. Try making that with your character creation sliders, eh? I’m not exactly a stranger to being somewhat less than represented in digital representations of people and bodies.
Back in , though, I had a thought: how easy would it be to get one of the more commonplace image generation models to generate me — or, at least, the parts of me that I’d never gotten a computer to faithfully reproduce? And so I set out to try and explain my arms and hands to DALL-E and Midjourney. Could other models have done better? I’m sure they could, especially months later, but I was aiming for some of the tools that were most commonplace and discussed, the tools that were in the hands of the masses the most at the time.
The results were, as you’ve probably guessed, disappointing. DALL-E surfaced standard-issue arms and hands. Midjourney showed veiny, muscular arms with hands that seemed painfully clasped together, but still, the anatomy was standard-issue. Thumbs, too, were inescapable. I tried again several times, rewording the prompts and making them more specific, all to no avail. If I was gonna make my arms happen, it certainly wasn’t going to come easily.
It’s easy to point to a technical reason for this experience. Bodies like mine are incredibly uncommon, and the probability that anyone who looks like me found their way into the AIs’ training data is very unlikely, let alone in a well-described, well-classified way. I am the spotless giraffe.
In theory, expanding the inclusion of less normative bodies in the training data would push the needle towards making it easier to create facsimiles of me.
…Except.
Tech is not neutral. It can’t be. It is always the sum total of human decisions, priorities, and tradeoffs, deployed to meet certain ends and desires, and particularly capitalistic interests. AI is far from being an exception to the rule. And in this case, any desire for image generation models to be able to represent me is going to butt heads with another incentive: the desire to avoid shocking users with body horror.
When DALL-E, Midjourney, and other image generation tools entered the scene, they were laughably bad at rendering human limbs. AI-rendered human bodies in general veered heavily into the uncanny valley, and generators produced straight-up body horror pretty often. And as it turns out, having a reputation for inaccurate anatomy and surprise shock value is bad for business!
Successive model retrainings have made rendering humans much more accurate, and tighter restrictions on prompts have made it much harder to generate body horror, even intentionally. As a consequence, non-normative bodies are also incredibly difficult to generate, even when the engine is fed hyperspecific prompts. When I’ve tried, I’ve gotten the sense the generators are pushing back against me. Still thinking about that a few months later, I posted:
– Ben Myers (@ben@a11y.info)
One of these days, we’ll reckon with the fact that AI art generators’ anti-body-horror filters also make it downright impossible to generate a person that looks like me.
It’s not just that the training sets simply don’t have examples of people who look like me. It’s that the system is now explicitly engineered to resist imagining me.
…Hey, is now a good a time to mention that in an effort to create a welcoming and inclusive community for all users,
the Midjourney Community Guidelines consider deformed bodies
a form of gore, and thus forbidden?
It is something of an amusing curiosity that some AI models were perplexed by a giraffe without spots. But it’s these same tools and paradigms that enshrine normativity of all kinds, sanding away the unusual.
As tech continues to charge headfirst into AI hype, this is going to have far-reaching, yet largely invisible to the mainstream, consequences to anyone on the wrong side of that normativity. Better hope you have spots.